Max level shown:
1 [ Mar 4 03:17:43 Enabled. ]
2 [ Mar 4 03:17:43 Rereading configuration. ]
3 [ Mar 4 03:17:50 Rereading configuration. ]
4 [ Mar 4 03:18:02 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/omicron-nexus/bin/nexus /var/svc/manifest/site/nexus/config.toml &"). ]
5 [ Mar 4 03:18:02 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72025-03-04T03:18:02.451ZDEBGnexus: registered DTrace probes
82025-03-04T03:18:02.453ZINFOnexus: setting up nexus server file = nexus/src/lib.rs:78
92025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Action file = nexus/auth/src/authz/oso_generic.rs:68
102025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = AnyActor file = nexus/auth/src/authz/oso_generic.rs:68
112025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = AuthenticatedActor file = nexus/auth/src/authz/oso_generic.rs:68
122025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = BlueprintConfig file = nexus/auth/src/authz/oso_generic.rs:68
132025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Database file = nexus/auth/src/authz/oso_generic.rs:68
142025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = DnsConfig file = nexus/auth/src/authz/oso_generic.rs:68
152025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Fleet file = nexus/auth/src/authz/oso_generic.rs:68
162025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Inventory file = nexus/auth/src/authz/oso_generic.rs:68
172025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = IpPoolList file = nexus/auth/src/authz/oso_generic.rs:68
182025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = ConsoleSessionList file = nexus/auth/src/authz/oso_generic.rs:68
192025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequestList file = nexus/auth/src/authz/oso_generic.rs:68
202025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SiloCertificateList file = nexus/auth/src/authz/oso_generic.rs:68
212025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SiloIdentityProviderList file = nexus/auth/src/authz/oso_generic.rs:68
222025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SiloUserList file = nexus/auth/src/authz/oso_generic.rs:68
232025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Project file = nexus/auth/src/authz/oso_generic.rs:68
242025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Disk file = nexus/auth/src/authz/oso_generic.rs:68
252025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Snapshot file = nexus/auth/src/authz/oso_generic.rs:68
262025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = ProjectImage file = nexus/auth/src/authz/oso_generic.rs:68
272025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = AffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
282025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = AntiAffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
292025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Instance file = nexus/auth/src/authz/oso_generic.rs:68
302025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = IpPool file = nexus/auth/src/authz/oso_generic.rs:68
312025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = InstanceNetworkInterface file = nexus/auth/src/authz/oso_generic.rs:68
322025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Vpc file = nexus/auth/src/authz/oso_generic.rs:68
332025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = VpcRouter file = nexus/auth/src/authz/oso_generic.rs:68
342025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = InternetGateway file = nexus/auth/src/authz/oso_generic.rs:68
352025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpPool file = nexus/auth/src/authz/oso_generic.rs:68
362025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpAddress file = nexus/auth/src/authz/oso_generic.rs:68
372025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = RouterRoute file = nexus/auth/src/authz/oso_generic.rs:68
382025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = VpcSubnet file = nexus/auth/src/authz/oso_generic.rs:68
392025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = FloatingIp file = nexus/auth/src/authz/oso_generic.rs:68
402025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Image file = nexus/auth/src/authz/oso_generic.rs:68
412025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SiloImage file = nexus/auth/src/authz/oso_generic.rs:68
422025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = AddressLot file = nexus/auth/src/authz/oso_generic.rs:68
432025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Blueprint file = nexus/auth/src/authz/oso_generic.rs:68
442025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = LoopbackAddress file = nexus/auth/src/authz/oso_generic.rs:68
452025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Certificate file = nexus/auth/src/authz/oso_generic.rs:68
462025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = ConsoleSession file = nexus/auth/src/authz/oso_generic.rs:68
472025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequest file = nexus/auth/src/authz/oso_generic.rs:68
482025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = DeviceAccessToken file = nexus/auth/src/authz/oso_generic.rs:68
492025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = PhysicalDisk file = nexus/auth/src/authz/oso_generic.rs:68
502025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Rack file = nexus/auth/src/authz/oso_generic.rs:68
512025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = RoleBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
522025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SshKey file = nexus/auth/src/authz/oso_generic.rs:68
532025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Silo file = nexus/auth/src/authz/oso_generic.rs:68
542025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SiloUser file = nexus/auth/src/authz/oso_generic.rs:68
552025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SiloGroup file = nexus/auth/src/authz/oso_generic.rs:68
562025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SupportBundle file = nexus/auth/src/authz/oso_generic.rs:68
572025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = IdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
582025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = SamlIdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
592025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Sled file = nexus/auth/src/authz/oso_generic.rs:68
602025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = TufRepo file = nexus/auth/src/authz/oso_generic.rs:68
612025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = TufArtifact file = nexus/auth/src/authz/oso_generic.rs:68
622025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Zpool file = nexus/auth/src/authz/oso_generic.rs:68
632025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = Service file = nexus/auth/src/authz/oso_generic.rs:68
642025-03-04T03:18:02.459ZINFOnexus (ServerContext): registering Oso class class = UserBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
652025-03-04T03:18:02.459ZINFOnexus (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n file = nexus/auth/src/authz/oso_generic.rs:89
662025-03-04T03:18:02.479ZINFOnexus (ServerContext): Setting up resolver using DNS servers for subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344::, width: 48 } } file = nexus/src/context.rs:222
672025-03-04T03:18:02.481ZINFOnexus (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
682025-03-04T03:18:02.484ZINFOnexus (ServerContext): Setting up qorb database pool from DNS dns_addrs = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = nexus/src/context.rs:278
692025-03-04T03:18:02.490ZDEBGnexus (ServerContext): registered USDT probes
702025-03-04T03:18:02.823ZINFOnexus: Database schema version is up to date desired_version = 128.0.0 file = nexus/db-queries/src/db/datastore/db_metadata.rs:144 found_version = 128.0.0
712025-03-04T03:18:02.827ZINFOnexus (SEC): SEC running file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:813 sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
722025-03-04T03:18:02.827ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 995, tv_nsec: 26727874 } })
732025-03-04T03:18:02.828ZINFOnexus (Nexus): Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1155
742025-03-04T03:18:02.916ZINFOnexus (Nexus): determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
752025-03-04T03:18:02.916ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
762025-03-04T03:18:02.919ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcaba2c9-6e5b-46a4-bf9f-4ff9076ce3bd", "content-length": "26", "date": "Tue, 04 Mar 2025 03:18:02 GMT"} })
772025-03-04T03:18:02.919ZINFOnexus (Nexus): identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
782025-03-04T03:18:02.921ZINFOnexus (Nexus): completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
792025-03-04T03:18:02.953ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 995, tv_nsec: 152482055 } })
802025-03-04T03:18:02.953ZINFOnexus (Nexus): Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1155
812025-03-04T03:18:02.986ZINFOnexus (Nexus): determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
822025-03-04T03:18:02.986ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
832025-03-04T03:18:02.986ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "635eeae1-4f7b-4300-8e8e-21386dfff8f9", "content-length": "26", "date": "Tue, 04 Mar 2025 03:18:02 GMT"} })
842025-03-04T03:18:02.986ZINFOnexus (Nexus): identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
852025-03-04T03:18:02.987ZINFOnexus (Nexus): completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
862025-03-04T03:18:03.056ZDEBGnexus (clickhouse-client): registered USDT probes id = 6cd0c5bf-b2c8-4027-8278-b708e5b0d50d
872025-03-04T03:18:03.077ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
882025-03-04T03:18:03.088ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
892025-03-04T03:18:03.091ZDEBGnexus (DataLoader): attempting to create built-in users
902025-03-04T03:18:03.091ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
912025-03-04T03:18:03.092ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
922025-03-04T03:18:03.115ZINFOnexus (DataLoader): created 0 built-in users file = nexus/db-queries/src/db/datastore/silo_user.rs:394
932025-03-04T03:18:03.115ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
942025-03-04T03:18:03.116ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /bgtasks versions = all
952025-03-04T03:18:03.116ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /bgtasks/activate versions = all
962025-03-04T03:18:03.116ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /bgtasks/view/{bgtask_name} versions = all
972025-03-04T03:18:03.116ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /clickhouse/policy versions = all
982025-03-04T03:18:03.116ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /clickhouse/policy versions = all
992025-03-04T03:18:03.116ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
1002025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
1012025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
1022025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1032025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1042025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /demo-saga versions = all
1052025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1062025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/all versions = all
1072025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1082025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1092025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/import versions = all
1102025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/regenerate versions = all
1112025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/target versions = all
1122025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/target versions = all
1132025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /deployment/blueprints/target/enabled versions = all
1142025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1152025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /disks/{disk_id} versions = all
1162025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /instances/{instance_id}/migrate versions = all
1172025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /metrics/collectors versions = all
1182025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1192025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /metrics/producers versions = all
1202025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1212025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /physical-disk/expunge versions = all
1222025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /probes/{sled} versions = all
1232025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1242025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sagas versions = all
1252025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sagas/{saga_id} versions = all
1262025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sled-agents/{sled_id} versions = all
1272025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sled-agents/{sled_id} versions = all
1282025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1292025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sleds/add versions = all
1302025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sleds/expunge versions = all
1312025-03-04T03:18:03.117ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sleds/uninitialized versions = all
1322025-03-04T03:18:03.118ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /switch/{switch_id} versions = all
1332025-03-04T03:18:03.118ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /vmms/{propolis_id} versions = all
1342025-03-04T03:18:03.118ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1352025-03-04T03:18:03.118ZINFOnexus (dropshot_internal): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:12221
1362025-03-04T03:18:03.118ZDEBGnexus (dropshot_internal): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12221
1372025-03-04T03:18:03.118ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1382025-03-04T03:18:03.118ZDEBGnexus (DataLoader): attempting to create built-in roles
1392025-03-04T03:18:03.118ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1402025-03-04T03:18:03.118ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1412025-03-04T03:18:03.118ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1422025-03-04T03:18:03.118ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1432025-03-04T03:18:03.120ZWARNnexus: Cannot look up rack: Object (of type ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05)) not found: rack file = nexus/src/app/rack.rs:776
1442025-03-04T03:18:03.255ZINFOnexus (DataLoader): created 0 built-in roles file = nexus/db-queries/src/db/datastore/role.rs:84
1452025-03-04T03:18:03.255ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1462025-03-04T03:18:03.257ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1472025-03-04T03:18:03.257ZDEBGnexus (DataLoader): attempting to create built-in role assignments
1482025-03-04T03:18:03.257ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1492025-03-04T03:18:03.257ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1502025-03-04T03:18:03.552ZINFOnexus (DataLoader): created 4 built-in role assignments file = nexus/db-queries/src/db/datastore/role.rs:113
1512025-03-04T03:18:03.552ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1522025-03-04T03:18:03.553ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1532025-03-04T03:18:03.553ZDEBGnexus (DataLoader): attempting to create built-in silos
1542025-03-04T03:18:03.553ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1552025-03-04T03:18:03.553ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1562025-03-04T03:18:04.491ZINFOnexus (DataLoader): created 2 built-in silos file = nexus/db-queries/src/db/datastore/silo.rs:93
1572025-03-04T03:18:04.491ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1582025-03-04T03:18:04.492ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1592025-03-04T03:18:04.551ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1602025-03-04T03:18:04.552ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1612025-03-04T03:18:04.552ZDEBGnexus (DataLoader): attempting to create built-in projects
1622025-03-04T03:18:04.553ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1632025-03-04T03:18:04.553ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1642025-03-04T03:18:04.739ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1652025-03-04T03:18:04.739ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1662025-03-04T03:18:04.749ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1672025-03-04T03:18:04.750ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1682025-03-04T03:18:04.784ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1692025-03-04T03:18:04.786ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1702025-03-04T03:18:04.786ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1712025-03-04T03:18:04.786ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1722025-03-04T03:18:04.821ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1732025-03-04T03:18:04.822ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1742025-03-04T03:18:05.120ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1752025-03-04T03:18:05.121ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1762025-03-04T03:18:05.123ZWARNnexus: Cannot look up rack: Object (of type ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05)) not found: rack file = nexus/src/app/rack.rs:776
1772025-03-04T03:18:06.462ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1782025-03-04T03:18:06.463ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1792025-03-04T03:18:06.463ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1802025-03-04T03:18:06.464ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1812025-03-04T03:18:06.662ZINFOnexus (DataLoader): created built-in services project file = nexus/db-queries/src/db/datastore/project.rs:122
1822025-03-04T03:18:06.662ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1832025-03-04T03:18:06.663ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1842025-03-04T03:18:06.663ZDEBGnexus (DataLoader): attempting to create built-in VPCs
1852025-03-04T03:18:06.663ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1862025-03-04T03:18:06.663ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1872025-03-04T03:18:06.700ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1882025-03-04T03:18:06.700ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1892025-03-04T03:18:06.761ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1902025-03-04T03:18:06.761ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1912025-03-04T03:18:06.762ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1922025-03-04T03:18:06.763ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1932025-03-04T03:18:06.827ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1942025-03-04T03:18:06.827ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1952025-03-04T03:18:06.828ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1962025-03-04T03:18:06.831ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
1972025-03-04T03:18:06.831ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1982025-03-04T03:18:06.831ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1992025-03-04T03:18:06.833ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2002025-03-04T03:18:06.833ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2012025-03-04T03:18:06.834ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2022025-03-04T03:18:06.834ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2032025-03-04T03:18:06.836ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2042025-03-04T03:18:06.838ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2052025-03-04T03:18:06.839ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2062025-03-04T03:18:06.839ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2072025-03-04T03:18:06.981ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2082025-03-04T03:18:06.982ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2092025-03-04T03:18:06.983ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2102025-03-04T03:18:06.983ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2112025-03-04T03:18:06.985ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2122025-03-04T03:18:06.985ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2132025-03-04T03:18:06.986ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2142025-03-04T03:18:06.989ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2152025-03-04T03:18:06.989ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2162025-03-04T03:18:06.989ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2172025-03-04T03:18:07.047ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2182025-03-04T03:18:07.047ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2192025-03-04T03:18:07.092ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2202025-03-04T03:18:07.092ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2212025-03-04T03:18:07.115ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2222025-03-04T03:18:07.116ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2232025-03-04T03:18:07.117ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2242025-03-04T03:18:07.117ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2252025-03-04T03:18:07.119ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2262025-03-04T03:18:07.119ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2272025-03-04T03:18:07.120ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2282025-03-04T03:18:07.123ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2292025-03-04T03:18:07.123ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2302025-03-04T03:18:07.123ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2312025-03-04T03:18:07.123ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
2322025-03-04T03:18:07.124ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
2332025-03-04T03:18:07.125ZWARNnexus: Cannot look up rack: Object (of type ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05)) not found: rack file = nexus/src/app/rack.rs:776
2342025-03-04T03:18:07.198ZERROnexus (DataLoader): gave up trying to populate built-in PopulateBuiltinVpcs error_message = InternalError { internal_message: "Unknown diesel error creating VpcRouter called \\"system\\": Record not found" } file = nexus/src/populate.rs:126
2352025-03-04T03:18:07.198ZERROnexus: populate failed file = nexus/src/app/mod.rs:548
2362025-03-04T03:18:07.198ZINFOnexus: activating background tasks file = nexus/src/app/mod.rs:554
2372025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
2382025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
2392025-03-04T03:18:07.201ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
2402025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
2412025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
2422025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
2432025-03-04T03:18:07.201ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
2442025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2452025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
2462025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2472025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
2482025-03-04T03:18:07.201ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:08:07.201105286 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
2492025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
2502025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
2512025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
2522025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2532025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2542025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2552025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2562025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2572025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2582025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
2592025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
2602025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2612025-03-04T03:18:07.201ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 95.011\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
2622025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 51.52\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
2632025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2642025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
2652025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
2662025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
2672025-03-04T03:18:07.203ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
2682025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
2692025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
2702025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
2712025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2722025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
2732025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2742025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
2752025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
2762025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2772025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2782025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2792025-03-04T03:18:07.203ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2802025-03-04T03:18:07.206ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
2812025-03-04T03:18:07.206ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2822025-03-04T03:18:07.206ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 999, tv_nsec: 402878867 } })
2832025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
2842025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
2852025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
2862025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2872025-03-04T03:18:07.206ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 999, tv_nsec: 403717518 } })
2882025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2892025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
2902025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
2912025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
2922025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
2932025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
2942025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
2952025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
2962025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2972025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
2982025-03-04T03:18:07.206ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector file = nexus/src/app/background/tasks/crdb_node_id_collector.rs:78 reason = no blueprint
2992025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3002025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3012025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 4.87847ms iteration = 1 status = {"error":"no blueprint"}
3022025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3032025-03-04T03:18:07.206ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
3042025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3052025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 4.862239ms iteration = 1 status = {"error":"no blueprint"}
3062025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3072025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3082025-03-04T03:18:07.206ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
3092025-03-04T03:18:07.206ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:71 reason = no blueprint
3102025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 4.89643ms iteration = 1 status = {"error":"no blueprint"}
3112025-03-04T03:18:07.206ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
3122025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3132025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3142025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3152025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
3162025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
3172025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
3182025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3192025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
3202025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
3212025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
3222025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3232025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
3242025-03-04T03:18:07.206ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
3252025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
3262025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3272025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3282025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3292025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
3302025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3312025-03-04T03:18:07.207ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external dns_group = external file = nexus/src/app/background/tasks/dns_servers.rs:113
3322025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3332025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
3342025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.407559ms iteration = 1 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
3352025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
3362025-03-04T03:18:07.207ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
3372025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 3.24\u{b5}s iteration = 2 status = {"error":"no config"}
3382025-03-04T03:18:07.207ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal dns_group = internal file = nexus/src/app/background/tasks/dns_servers.rs:113
3392025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.46628ms iteration = 1 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
3402025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
3412025-03-04T03:18:07.207ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
3422025-03-04T03:18:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.62\u{b5}s iteration = 2 status = {"error":"no config"}
3432025-03-04T03:18:07.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3442025-03-04T03:18:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3452025-03-04T03:18:07.208ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3462025-03-04T03:18:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3472025-03-04T03:18:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3482025-03-04T03:18:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3492025-03-04T03:18:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3502025-03-04T03:18:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3512025-03-04T03:18:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3522025-03-04T03:18:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3532025-03-04T03:18:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3542025-03-04T03:18:07.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3552025-03-04T03:18:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3562025-03-04T03:18:07.210ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3572025-03-04T03:18:07.211ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
3582025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3592025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3602025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3612025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3622025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3632025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3642025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
3652025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3662025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3672025-03-04T03:18:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3682025-03-04T03:18:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3692025-03-04T03:18:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3702025-03-04T03:18:07.212ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
3712025-03-04T03:18:07.212ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
3722025-03-04T03:18:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
3732025-03-04T03:18:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3742025-03-04T03:18:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
3752025-03-04T03:18:07.215ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3762025-03-04T03:18:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3772025-03-04T03:18:07.217ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3782025-03-04T03:18:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3792025-03-04T03:18:07.219ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3802025-03-04T03:18:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3812025-03-04T03:18:07.234ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3822025-03-04T03:18:07.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3832025-03-04T03:18:07.247ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3842025-03-04T03:18:07.248ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:08:07.201105286 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
3852025-03-04T03:18:07.248ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 47.687284ms iteration = 1 status = {"expiration":"2025-03-04T03:08:07.201105286Z","pruned":[]}
3862025-03-04T03:18:07.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3872025-03-04T03:18:07.251ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3882025-03-04T03:18:07.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3892025-03-04T03:18:07.262ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3902025-03-04T03:18:07.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3912025-03-04T03:18:07.265ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3922025-03-04T03:18:07.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3932025-03-04T03:18:07.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3942025-03-04T03:18:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
3952025-03-04T03:18:07.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3962025-03-04T03:18:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3972025-03-04T03:18:07.322ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 999, tv_nsec: 521845495 } })
3982025-03-04T03:18:07.323ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
3992025-03-04T03:18:07.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4002025-03-04T03:18:07.324ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4012025-03-04T03:18:07.325ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4022025-03-04T03:18:07.325ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4032025-03-04T03:18:07.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4042025-03-04T03:18:07.325ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 118.897527ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
4052025-03-04T03:18:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4062025-03-04T03:18:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4072025-03-04T03:18:07.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4082025-03-04T03:18:07.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4092025-03-04T03:18:07.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4102025-03-04T03:18:07.327ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4112025-03-04T03:18:07.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4122025-03-04T03:18:07.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4132025-03-04T03:18:07.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4142025-03-04T03:18:07.339ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4152025-03-04T03:18:07.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
4162025-03-04T03:18:07.341ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
4172025-03-04T03:18:07.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
4182025-03-04T03:18:07.343ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4192025-03-04T03:18:07.345ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4202025-03-04T03:18:07.347ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:80 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
4212025-03-04T03:18:07.347ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 146.082791ms iteration = 1 status = {"error":"rack not yet initialized: accfd3a5-958c-4905-8c5c-5f2ac43dda05"}
4222025-03-04T03:18:07.349ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4232025-03-04T03:18:07.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4242025-03-04T03:18:07.359ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4252025-03-04T03:18:07.360ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
4262025-03-04T03:18:07.360ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4272025-03-04T03:18:07.360ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "858ea3cc-c795-4e7a-b9fd-0bef976f818a", "content-length": "26", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
4282025-03-04T03:18:07.361ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4292025-03-04T03:18:07.361ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4302025-03-04T03:18:07.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4312025-03-04T03:18:07.361ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4322025-03-04T03:18:07.362ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4332025-03-04T03:18:07.369ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 999, tv_nsec: 568693760 } })
4342025-03-04T03:18:07.370ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
4352025-03-04T03:18:07.370ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4362025-03-04T03:18:07.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4372025-03-04T03:18:07.384ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 177.399954ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
4382025-03-04T03:18:07.386ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4392025-03-04T03:18:07.387ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
4402025-03-04T03:18:07.388ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 181.802609ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
4412025-03-04T03:18:07.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4422025-03-04T03:18:07.388ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 184.182677ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
4432025-03-04T03:18:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
4442025-03-04T03:18:07.389ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
4452025-03-04T03:18:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
4462025-03-04T03:18:07.389ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 188.160036ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4472025-03-04T03:18:07.390ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 189.024276ms iteration = 1 status = {}
4482025-03-04T03:18:07.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4492025-03-04T03:18:07.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4502025-03-04T03:18:07.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4512025-03-04T03:18:07.392ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4522025-03-04T03:18:07.392ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4532025-03-04T03:18:07.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4542025-03-04T03:18:07.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4552025-03-04T03:18:07.394ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4562025-03-04T03:18:07.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4572025-03-04T03:18:07.395ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4582025-03-04T03:18:07.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4592025-03-04T03:18:07.401ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4602025-03-04T03:18:07.401ZWARNnexus (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker file = nexus/src/app/background/tasks/sync_service_zone_nat.rs:95
4612025-03-04T03:18:07.401ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 200.074002ms iteration = 1 status = {"error":"inventory collection is None"}
4622025-03-04T03:18:07.402ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccc37713-77ee-46a8-9f94-7d625073846b", "content-length": "1", "date": "Tue, 04 Mar 2025 03:18:06 GMT"} })
4632025-03-04T03:18:07.403ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4642025-03-04T03:18:07.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4652025-03-04T03:18:07.403ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4662025-03-04T03:18:07.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4672025-03-04T03:18:07.404ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4682025-03-04T03:18:07.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4692025-03-04T03:18:07.409ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4702025-03-04T03:18:07.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4712025-03-04T03:18:07.414ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4722025-03-04T03:18:07.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4732025-03-04T03:18:07.415ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
4742025-03-04T03:18:07.415ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 214.325237ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
4752025-03-04T03:18:07.415ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
4762025-03-04T03:18:07.415ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4772025-03-04T03:18:07.416ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "84b6ae3d-94a6-4e6a-b53a-2641bcd91c9d", "content-length": "26", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
4782025-03-04T03:18:07.416ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4792025-03-04T03:18:07.416ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4802025-03-04T03:18:07.417ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4812025-03-04T03:18:07.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4822025-03-04T03:18:07.426ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4832025-03-04T03:18:07.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4842025-03-04T03:18:07.435ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4852025-03-04T03:18:07.436ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 233.667874ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4862025-03-04T03:18:07.438ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4872025-03-04T03:18:07.438ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4882025-03-04T03:18:07.438ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4892025-03-04T03:18:07.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4902025-03-04T03:18:07.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4912025-03-04T03:18:07.441ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4922025-03-04T03:18:07.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4932025-03-04T03:18:07.446ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4942025-03-04T03:18:07.450ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 249.085693ms iteration = 1 status = null
4952025-03-04T03:18:07.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4962025-03-04T03:18:07.450ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4972025-03-04T03:18:07.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4982025-03-04T03:18:07.450ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
4992025-03-04T03:18:07.450ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 249.660489ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
5002025-03-04T03:18:07.454ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
5012025-03-04T03:18:07.454ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 252.828378ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
5022025-03-04T03:18:07.458ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5032025-03-04T03:18:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5042025-03-04T03:18:07.465ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 264.643343ms iteration = 1 status = {}
5052025-03-04T03:18:07.474ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5062025-03-04T03:18:07.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5072025-03-04T03:18:07.488ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
5082025-03-04T03:18:07.489ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 999, tv_nsec: 688391066 } })
5092025-03-04T03:18:07.496ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5102025-03-04T03:18:07.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5112025-03-04T03:18:07.515ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5122025-03-04T03:18:07.515ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 312.767043ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5132025-03-04T03:18:07.521ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone file = nexus/src/app/background/tasks/external_endpoints.rs:48
5142025-03-04T03:18:07.521ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 320.168944ms iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
5152025-03-04T03:18:07.523ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5162025-03-04T03:18:07.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5172025-03-04T03:18:07.524ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
5182025-03-04T03:18:07.524ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
5192025-03-04T03:18:07.524ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 322.838277ms iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
5202025-03-04T03:18:07.525ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5212025-03-04T03:18:07.527ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
5222025-03-04T03:18:07.527ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
5232025-03-04T03:18:07.527ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 322.30865ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
5242025-03-04T03:18:07.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
5252025-03-04T03:18:07.527ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5262025-03-04T03:18:07.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5272025-03-04T03:18:07.528ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
5282025-03-04T03:18:07.528ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
5292025-03-04T03:18:07.528ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
5302025-03-04T03:18:07.528ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "48d40560-0b1f-4176-8357-c12f78387089", "content-length": "476", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5312025-03-04T03:18:07.531ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
5322025-03-04T03:18:07.531ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a283fe34-92b6-4cd8-b2cd-de3194a943e4", "content-length": "734", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5332025-03-04T03:18:07.533ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
5342025-03-04T03:18:07.534ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6127572-f1b5-41e9-8fe7-74d10915ef4c", "content-length": "111", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5352025-03-04T03:18:07.534ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
5362025-03-04T03:18:07.534ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37f89a7d-825d-4c9a-a5c3-54cd638f9b9d", "content-length": "111", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5372025-03-04T03:18:07.534ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
5382025-03-04T03:18:07.535ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "102526b8-60fd-48e1-9468-9f40095ae5a5", "content-length": "167", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5392025-03-04T03:18:07.535ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
5402025-03-04T03:18:07.535ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd5f1b20-1374-4945-a8c7-4e09923b318c", "content-length": "167", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5412025-03-04T03:18:07.535ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
5422025-03-04T03:18:07.535ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5740e430-3c45-441c-b814-29664b2c394a", "content-length": "175", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5432025-03-04T03:18:07.535ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
5442025-03-04T03:18:07.536ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a79e7782-01b3-467f-8d93-aec5fff77e88", "content-length": "175", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5452025-03-04T03:18:07.536ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
5462025-03-04T03:18:07.537ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6988808a-59ca-4ecc-9d7b-fed97a556e0f", "content-length": "702", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5472025-03-04T03:18:07.537ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5482025-03-04T03:18:07.538ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
5492025-03-04T03:18:07.538ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 335.143247ms iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
5502025-03-04T03:18:07.542ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d6d465c-fe33-4fa7-9769-493a234c1fee", "content-length": "727", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5512025-03-04T03:18:07.543ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5522025-03-04T03:18:07.543ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "725aa1eb-9e9e-4f67-9547-998b9523aa2e", "content-length": "729", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5532025-03-04T03:18:07.543ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5542025-03-04T03:18:07.543ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33dd9f97-fc4e-4ecf-a9ae-e43a7362593c", "content-length": "728", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5552025-03-04T03:18:07.543ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
5562025-03-04T03:18:07.543ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "febd6af1-e226-43bc-9a50-2da20b182b75", "content-length": "722", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5572025-03-04T03:18:07.543ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
5582025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3430c4e-e11f-4aa9-9d86-e0a25d8d5b5c", "content-length": "109", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5592025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
5602025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 342.532398ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5612025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5622025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ad3ed9b-49e9-4cdd-a08b-ac327211c679", "content-length": "109", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5632025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
5642025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5652025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9cac570c-d7a3-448a-a698-34f71800b324", "content-length": "169", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5662025-03-04T03:18:07.544ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
5672025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "11a8a290-f9cb-4070-89d8-3142b4c2cacf", "content-length": "169", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5682025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
5692025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5702025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 338.154604ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
5712025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5722025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeeb501b-45b5-4f31-8f79-12c527f42887", "content-length": "178", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5732025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
5742025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "501c6164-1c97-4748-b411-358caab7f84b", "content-length": "178", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5752025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
5762025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "020c7215-b17e-4618-929d-93a1719259fd", "content-length": "702", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5772025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
5782025-03-04T03:18:07.545ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5792025-03-04T03:18:07.546ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffc655a9-7d72-4a93-94d4-54151936d0d3", "content-length": "727", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5802025-03-04T03:18:07.546ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
5812025-03-04T03:18:07.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5822025-03-04T03:18:07.546ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0717ac5d-8986-4ba0-9c7e-5643963fd9d7", "content-length": "729", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5832025-03-04T03:18:07.546ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
5842025-03-04T03:18:07.546ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "507251b8-2f51-4b95-b9dd-a786cc471b50", "content-length": "728", "date": "Tue, 04 Mar 2025 03:18:07 GMT"} })
5852025-03-04T03:18:07.546ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
5862025-03-04T03:18:07.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
5872025-03-04T03:18:07.548ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5882025-03-04T03:18:07.548ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5892025-03-04T03:18:07.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5902025-03-04T03:18:07.549ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
5912025-03-04T03:18:07.549ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
5922025-03-04T03:18:07.549ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5932025-03-04T03:18:07.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
5942025-03-04T03:18:07.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5952025-03-04T03:18:07.555ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5962025-03-04T03:18:07.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5972025-03-04T03:18:07.557ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5982025-03-04T03:18:07.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5992025-03-04T03:18:07.561ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 354.08843ms iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
6002025-03-04T03:18:07.565ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 362.868707ms iteration = 1 status = {}
6012025-03-04T03:18:07.573ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6022025-03-04T03:18:07.573ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6032025-03-04T03:18:07.573ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6042025-03-04T03:18:07.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6052025-03-04T03:18:07.578ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 371.056507ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6062025-03-04T03:18:07.585ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6072025-03-04T03:18:07.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
6082025-03-04T03:18:07.587ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6092025-03-04T03:18:07.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6102025-03-04T03:18:07.589ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {} vpc_interfaces = {}
6112025-03-04T03:18:07.590ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6122025-03-04T03:18:07.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6132025-03-04T03:18:07.592ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6142025-03-04T03:18:07.593ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
6152025-03-04T03:18:07.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
6162025-03-04T03:18:07.593ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
6172025-03-04T03:18:07.593ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6182025-03-04T03:18:07.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6192025-03-04T03:18:07.594ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6202025-03-04T03:18:07.596ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
6212025-03-04T03:18:07.596ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
6222025-03-04T03:18:07.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
6232025-03-04T03:18:07.607ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
6242025-03-04T03:18:07.607ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 405.416489ms iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
6252025-03-04T03:18:07.628ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
6262025-03-04T03:18:07.628ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
6272025-03-04T03:18:07.628ZDEBGnexus (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
6282025-03-04T03:18:07.630ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 423.383898ms iteration = 1 status = {"last_run_counters":{"copy_err":0,"copy_ok":0,"delete_err":0,"delete_ok":0,"list_err":0,"list_ok":0,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"delete_err":0,"delete_ok":0,"list_err":0,"list_ok":0,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[]}
6292025-03-04T03:18:07.646ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6302025-03-04T03:18:07.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6312025-03-04T03:18:07.669ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6322025-03-04T03:18:07.669ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6332025-03-04T03:18:07.669ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 466.814681ms iteration = 1 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6342025-03-04T03:18:07.769ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
6352025-03-04T03:18:07.769ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
6362025-03-04T03:18:07.769ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
6372025-03-04T03:18:07.770ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6382025-03-04T03:18:07.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6392025-03-04T03:18:07.772ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6402025-03-04T03:18:07.773ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
6412025-03-04T03:18:07.773ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6422025-03-04T03:18:07.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6432025-03-04T03:18:07.887ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6442025-03-04T03:18:07.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6452025-03-04T03:18:07.889ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6462025-03-04T03:18:07.890ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6472025-03-04T03:18:07.890ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6482025-03-04T03:18:07.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6492025-03-04T03:18:07.892ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6502025-03-04T03:18:07.893ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6512025-03-04T03:18:07.893ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6522025-03-04T03:18:07.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6532025-03-04T03:18:07.899ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 698.312189ms iteration = 1 status = {}
6542025-03-04T03:18:07.974ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
6552025-03-04T03:18:07.974ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
6562025-03-04T03:18:07.974ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
6572025-03-04T03:18:07.975ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 768.489108ms
6582025-03-04T03:18:07.975ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 771.93431ms iteration = 1 status = {}
6592025-03-04T03:18:08.256ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 60c90db3-1618-44fd-a217-8a46e7ca2619 file = nexus/db-queries/src/db/datastore/inventory.rs:1010
6602025-03-04T03:18:08.256ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 60c90db3-1618-44fd-a217-8a46e7ca2619 time_started = 2025-03-04 03:18:07.527467 UTC
6612025-03-04T03:18:08.256ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.054690526s iteration = 1 status = {"collection_id":"60c90db3-1618-44fd-a217-8a46e7ca2619","time_done":"2025-03-04 03:18:07.769520 UTC","time_started":"2025-03-04 03:18:07.527467 UTC"}
6622025-03-04T03:18:08.256ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
6632025-03-04T03:18:08.256ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
6642025-03-04T03:18:08.256ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
6652025-03-04T03:18:08.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 21.97\u{b5}s iteration = 2 status = {"error":"no blueprint"}
6662025-03-04T03:18:08.256ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6672025-03-04T03:18:08.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6682025-03-04T03:18:08.258ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6692025-03-04T03:18:08.259ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6702025-03-04T03:18:08.259ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6712025-03-04T03:18:08.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6722025-03-04T03:18:08.262ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:80 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
6732025-03-04T03:18:08.262ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 6.318087ms iteration = 2 status = {"error":"rack not yet initialized: accfd3a5-958c-4905-8c5c-5f2ac43dda05"}
6742025-03-04T03:18:09.126ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6752025-03-04T03:18:09.126ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6762025-03-04T03:18:09.127ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6772025-03-04T03:18:09.128ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6782025-03-04T03:18:09.129ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6792025-03-04T03:18:09.132ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
6802025-03-04T03:18:09.132ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
6812025-03-04T03:18:11.133ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6822025-03-04T03:18:11.133ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6832025-03-04T03:18:11.134ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6842025-03-04T03:18:11.135ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6852025-03-04T03:18:11.136ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6862025-03-04T03:18:11.138ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
6872025-03-04T03:18:11.138ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
6882025-03-04T03:18:13.139ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6892025-03-04T03:18:13.140ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6902025-03-04T03:18:13.141ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6912025-03-04T03:18:13.141ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6922025-03-04T03:18:13.142ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6932025-03-04T03:18:13.145ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
6942025-03-04T03:18:13.145ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
6952025-03-04T03:18:15.146ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6962025-03-04T03:18:15.146ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6972025-03-04T03:18:15.148ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6982025-03-04T03:18:15.148ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6992025-03-04T03:18:15.150ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7002025-03-04T03:18:15.152ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
7012025-03-04T03:18:15.152ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
7022025-03-04T03:18:17.153ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7032025-03-04T03:18:17.153ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7042025-03-04T03:18:17.155ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7052025-03-04T03:18:17.155ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7062025-03-04T03:18:17.157ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7072025-03-04T03:18:17.159ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
7082025-03-04T03:18:17.159ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
7092025-03-04T03:18:17.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
7102025-03-04T03:18:17.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7112025-03-04T03:18:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7122025-03-04T03:18:17.202ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
7132025-03-04T03:18:17.202ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7142025-03-04T03:18:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7152025-03-04T03:18:17.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7162025-03-04T03:18:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7172025-03-04T03:18:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7182025-03-04T03:18:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7192025-03-04T03:18:17.207ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
7202025-03-04T03:18:17.207ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.535318ms iteration = 2 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
7212025-03-04T03:18:17.232ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7222025-03-04T03:18:17.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7232025-03-04T03:18:17.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 70.17448ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7242025-03-04T03:18:19.161ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7252025-03-04T03:18:19.161ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7262025-03-04T03:18:19.163ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7272025-03-04T03:18:19.163ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7282025-03-04T03:18:19.164ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7292025-03-04T03:18:19.166ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
7302025-03-04T03:18:19.167ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
7312025-03-04T03:18:21.168ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7322025-03-04T03:18:21.168ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7332025-03-04T03:18:21.170ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7342025-03-04T03:18:21.170ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7352025-03-04T03:18:21.171ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7362025-03-04T03:18:21.173ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
7372025-03-04T03:18:21.174ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
7382025-03-04T03:18:23.175ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7392025-03-04T03:18:23.175ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7402025-03-04T03:18:23.177ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7412025-03-04T03:18:23.177ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7422025-03-04T03:18:23.179ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7432025-03-04T03:18:23.181ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
7442025-03-04T03:18:23.181ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
7452025-03-04T03:18:25.183ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7462025-03-04T03:18:25.183ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7472025-03-04T03:18:25.184ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7482025-03-04T03:18:25.184ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7492025-03-04T03:18:25.186ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7502025-03-04T03:18:25.189ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
7512025-03-04T03:18:25.189ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
7522025-03-04T03:18:27.190ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7532025-03-04T03:18:27.190ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7542025-03-04T03:18:27.192ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7552025-03-04T03:18:27.192ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7562025-03-04T03:18:27.194ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7572025-03-04T03:18:27.196ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
7582025-03-04T03:18:27.196ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
7592025-03-04T03:18:27.201ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
7602025-03-04T03:18:27.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7612025-03-04T03:18:27.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7622025-03-04T03:18:27.202ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 3 reason = Timeout
7632025-03-04T03:18:27.202ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7642025-03-04T03:18:27.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7652025-03-04T03:18:27.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7662025-03-04T03:18:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7672025-03-04T03:18:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7682025-03-04T03:18:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7692025-03-04T03:18:27.208ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
7702025-03-04T03:18:27.208ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.900145ms iteration = 3 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
7712025-03-04T03:18:27.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7722025-03-04T03:18:27.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7732025-03-04T03:18:27.269ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.459895ms iteration = 3 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7742025-03-04T03:18:29.198ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7752025-03-04T03:18:29.198ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7762025-03-04T03:18:29.200ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7772025-03-04T03:18:29.200ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7782025-03-04T03:18:29.202ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7792025-03-04T03:18:29.204ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
7802025-03-04T03:18:29.204ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
7812025-03-04T03:18:30.632ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:62177
7822025-03-04T03:18:30.640ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 9591c4cc-d6df-4004-9011-3d7683f2db26 roles = RoleSet { roles: {} } uri = /sled-agents/5525e426-7299-43ab-ad2b-346968b24e1c
7832025-03-04T03:18:30.641ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 9591c4cc-d6df-4004-9011-3d7683f2db26 resource = Database result = Ok(()) uri = /sled-agents/5525e426-7299-43ab-ad2b-346968b24e1c
7842025-03-04T03:18:30.647ZINFOnexus (dropshot_internal): request completed error_message_external = not found: sled with id "5525e426-7299-43ab-ad2b-346968b24e1c" error_message_internal = not found: sled with id "5525e426-7299-43ab-ad2b-346968b24e1c" file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:855 latency_us = 15289 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 9591c4cc-d6df-4004-9011-3d7683f2db26 response_code = 404 uri = /sled-agents/5525e426-7299-43ab-ad2b-346968b24e1c
7852025-03-04T03:18:30.656ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 2bb0a884-dde4-4e02-91c9-10d822bcf7df roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
7862025-03-04T03:18:30.656ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 2bb0a884-dde4-4e02-91c9-10d822bcf7df resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
7872025-03-04T03:18:30.674ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 26531 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 2bb0a884-dde4-4e02-91c9-10d822bcf7df response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
7882025-03-04T03:18:30.785ZINFOnexus: registered sled agent file = nexus/src/app/sled.rs:58 sled_uuid = 5525e426-7299-43ab-ad2b-346968b24e1c
7892025-03-04T03:18:30.821ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 61012 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 8b836c05-b0b2-4792-bb3d-89a4d8eb3d0c response_code = 204 uri = /sled-agents/5525e426-7299-43ab-ad2b-346968b24e1c
7902025-03-04T03:18:30.821ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
7912025-03-04T03:18:30.821ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7922025-03-04T03:18:30.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7932025-03-04T03:18:30.823ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7942025-03-04T03:18:30.824ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
7952025-03-04T03:18:30.824ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7962025-03-04T03:18:30.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7972025-03-04T03:18:30.831ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(60c90db3-1618-44fd-a217-8a46e7ca2619, 0), (04891a77-4557-4e1d-91ec-73cedbb3de6c, 0), (69465f72-82eb-4096-b064-2a2e721c6f40, 0)]
7982025-03-04T03:18:30.832ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1023, tv_nsec: 31091151 } })
7992025-03-04T03:18:30.863ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
8002025-03-04T03:18:30.863ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
8012025-03-04T03:18:30.863ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
8022025-03-04T03:18:30.864ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "348d9429-7d04-4a07-81d8-717e605e4635", "content-length": "476", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8032025-03-04T03:18:30.864ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
8042025-03-04T03:18:30.864ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "372be65c-07a0-4ff5-9c14-a71321a780f5", "content-length": "734", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8052025-03-04T03:18:30.864ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
8062025-03-04T03:18:30.864ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c38f75ca-5829-41f2-9f3e-73fd41199fad", "content-length": "111", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8072025-03-04T03:18:30.864ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
8082025-03-04T03:18:30.864ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bc13a60-399f-44df-89c7-5348a94dc4ff", "content-length": "111", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8092025-03-04T03:18:30.864ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
8102025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8c769a8-97dc-4367-b8b1-462c4353c452", "content-length": "167", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8112025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
8122025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77eefdc2-fc65-40b7-940c-945393c0f63a", "content-length": "167", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8132025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
8142025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "df006b1f-4883-4c8c-8708-2e5606d534d2", "content-length": "175", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8152025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
8162025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8dfc35c-e6c0-4a9e-beec-46f6e839ab67", "content-length": "175", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8172025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
8182025-03-04T03:18:30.865ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d888d7a-ec77-4471-b932-0147c9031339", "content-length": "702", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8192025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8202025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b761d86-a7ce-4c09-897e-13db6cbbca53", "content-length": "727", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8212025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8222025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3054002-fc0e-4239-ab96-62d6393d195b", "content-length": "729", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8232025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8242025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e235a0a2-d883-4690-85af-e8f7cc12b712", "content-length": "728", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8252025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
8262025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d8841cf-2128-4304-8f9e-d37c7994a2d9", "content-length": "722", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8272025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
8282025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe41d972-dfb7-4a64-9012-68cc75e8ecc2", "content-length": "109", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8292025-03-04T03:18:30.866ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
8302025-03-04T03:18:30.867ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0db71fc-b22f-49b9-8192-409be072c5a9", "content-length": "109", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8312025-03-04T03:18:30.867ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
8322025-03-04T03:18:30.867ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3920c129-f7a7-4564-8d55-341bbf037801", "content-length": "169", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8332025-03-04T03:18:30.867ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
8342025-03-04T03:18:30.867ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e0e5964-e5b0-4956-8700-7b7c6e5c9625", "content-length": "169", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8352025-03-04T03:18:30.867ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
8362025-03-04T03:18:30.867ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9408b0fe-c86c-4ae3-8ece-e7d988e7da81", "content-length": "178", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8372025-03-04T03:18:30.867ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
8382025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29c91b38-7a6b-487b-93e7-596b38636fe9", "content-length": "178", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8392025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
8402025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ac65664-95b8-417c-9d36-656b22897faa", "content-length": "702", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8412025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8422025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c699be64-ec13-4d12-b9ff-b27edaa9b1dc", "content-length": "727", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8432025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8442025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8244792-f6a4-4ef7-8885-600d407b11c1", "content-length": "729", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8452025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8462025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0ad64c6-8d35-404b-a6c2-05b06baa30e2", "content-length": "728", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8472025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8482025-03-04T03:18:30.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8492025-03-04T03:18:30.870ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8502025-03-04T03:18:30.872ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
8512025-03-04T03:18:30.872ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8522025-03-04T03:18:30.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8532025-03-04T03:18:30.873ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8542025-03-04T03:18:30.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
8552025-03-04T03:18:30.875ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8562025-03-04T03:18:30.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8572025-03-04T03:18:30.909ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
8582025-03-04T03:18:30.909ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
8592025-03-04T03:18:31.116ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a89a159-f82a-4be8-b207-0e6924561dd8", "content-length": "31959", "date": "Tue, 04 Mar 2025 03:18:30 GMT"} })
8602025-03-04T03:18:31.140ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
8612025-03-04T03:18:31.140ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
8622025-03-04T03:18:31.140ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
8632025-03-04T03:18:31.141ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8642025-03-04T03:18:31.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8652025-03-04T03:18:31.143ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8662025-03-04T03:18:31.145ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8672025-03-04T03:18:31.149ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8682025-03-04T03:18:31.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8692025-03-04T03:18:31.205ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
8702025-03-04T03:18:31.206ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
8712025-03-04T03:18:31.206ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
8722025-03-04T03:18:31.207ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
8732025-03-04T03:18:31.208ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8742025-03-04T03:18:31.210ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
8752025-03-04T03:18:31.210ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
8762025-03-04T03:18:31.529ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = f43e3c30-d1e9-4487-b67f-c851f5bb40d3 file = nexus/db-queries/src/db/datastore/inventory.rs:1010
8772025-03-04T03:18:31.529ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = f43e3c30-d1e9-4487-b67f-c851f5bb40d3 time_started = 2025-03-04 03:18:30.863691 UTC
8782025-03-04T03:18:31.529ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 708.498474ms iteration = 2 status = {"collection_id":"f43e3c30-d1e9-4487-b67f-c851f5bb40d3","time_done":"2025-03-04 03:18:31.140545 UTC","time_started":"2025-03-04 03:18:30.863691 UTC"}
8792025-03-04T03:18:31.529ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Dependency
8802025-03-04T03:18:31.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 3 reason = Dependency
8812025-03-04T03:18:31.529ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
8822025-03-04T03:18:31.529ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 5.03\u{b5}s iteration = 3 status = {"error":"no blueprint"}
8832025-03-04T03:18:31.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8842025-03-04T03:18:31.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8852025-03-04T03:18:31.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8862025-03-04T03:18:31.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8872025-03-04T03:18:31.536ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8882025-03-04T03:18:31.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8892025-03-04T03:18:31.633ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:80 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
8902025-03-04T03:18:31.633ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 103.988754ms iteration = 3 status = {"error":"rack not yet initialized: accfd3a5-958c-4905-8c5c-5f2ac43dda05"}
8912025-03-04T03:18:31.695ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 4b087bba-daac-499e-a9a3-457ae9013872 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
8922025-03-04T03:18:31.696ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 4b087bba-daac-499e-a9a3-457ae9013872 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
8932025-03-04T03:18:31.698ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2875 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:62177 req_id = 4b087bba-daac-499e-a9a3-457ae9013872 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
8942025-03-04T03:18:31.704ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:63537
8952025-03-04T03:18:32.717ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f24ee49a-2a57-43a6-a6cf-205d476ab93d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
8962025-03-04T03:18:32.717ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f24ee49a-2a57-43a6-a6cf-205d476ab93d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
8972025-03-04T03:18:32.719ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2160 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f24ee49a-2a57-43a6-a6cf-205d476ab93d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
8982025-03-04T03:18:32.887ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:60484
8992025-03-04T03:18:32.887ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60484 req_id = 0aaf5adf-50ac-4dfb-9c7d-8fa0de08d468 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
9002025-03-04T03:18:32.887ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60484 req_id = 0aaf5adf-50ac-4dfb-9c7d-8fa0de08d468 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
9012025-03-04T03:18:32.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 5753 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60484 req_id = 0aaf5adf-50ac-4dfb-9c7d-8fa0de08d468 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
9022025-03-04T03:18:32.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60484 req_id = 3e85346c-c3cc-42e8-958b-713da0e69447 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4YWY1NmQwOS01YmZmLTQ4MTAtOGQ4Zi0wMmI0NzBlY2M2NzIifX0%3D
9032025-03-04T03:18:32.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60484 req_id = 3e85346c-c3cc-42e8-958b-713da0e69447 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4YWY1NmQwOS01YmZmLTQ4MTAtOGQ4Zi0wMmI0NzBlY2M2NzIifX0%3D
9042025-03-04T03:18:32.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3215 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60484 req_id = 3e85346c-c3cc-42e8-958b-713da0e69447 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4YWY1NmQwOS01YmZmLTQ4MTAtOGQ4Zi0wMmI0NzBlY2M2NzIifX0%3D
9052025-03-04T03:18:33.212ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9062025-03-04T03:18:33.212ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9072025-03-04T03:18:33.214ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9082025-03-04T03:18:33.214ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9092025-03-04T03:18:33.216ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9102025-03-04T03:18:33.218ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
9112025-03-04T03:18:33.218ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
9122025-03-04T03:18:33.232ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:41988
9132025-03-04T03:18:33.232ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 14b8434b-ac4c-42e7-ac8e-16cb171c6e6e roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
9142025-03-04T03:18:33.232ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 14b8434b-ac4c-42e7-ac8e-16cb171c6e6e resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
9152025-03-04T03:18:33.244ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 11778 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 14b8434b-ac4c-42e7-ac8e-16cb171c6e6e response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
9162025-03-04T03:18:33.245ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = dd911d14-fff8-4980-a317-d8b417e3a266 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
9172025-03-04T03:18:33.245ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = dd911d14-fff8-4980-a317-d8b417e3a266 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
9182025-03-04T03:18:33.269ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 23969 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = dd911d14-fff8-4980-a317-d8b417e3a266 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
9192025-03-04T03:18:33.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = ca3ca5d8-c898-4392-922b-4587becada0d roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
9202025-03-04T03:18:33.269ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = ca3ca5d8-c898-4392-922b-4587becada0d resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
9212025-03-04T03:18:33.286ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 16670 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = ca3ca5d8-c898-4392-922b-4587becada0d response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
9222025-03-04T03:18:33.286ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = cfc9d502-8621-4af5-b2aa-7e4108fd84e2 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
9232025-03-04T03:18:33.286ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = cfc9d502-8621-4af5-b2aa-7e4108fd84e2 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
9242025-03-04T03:18:33.305ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 19420 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = cfc9d502-8621-4af5-b2aa-7e4108fd84e2 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
9252025-03-04T03:18:33.306ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = c7083943-82ca-4e13-94ad-6dc34f6573c2 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
9262025-03-04T03:18:33.306ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = c7083943-82ca-4e13-94ad-6dc34f6573c2 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
9272025-03-04T03:18:33.316ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 10593 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = c7083943-82ca-4e13-94ad-6dc34f6573c2 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
9282025-03-04T03:18:33.316ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = f9d8d687-c155-45bc-9271-6c51e4e2bf1d roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
9292025-03-04T03:18:33.317ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = f9d8d687-c155-45bc-9271-6c51e4e2bf1d resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
9302025-03-04T03:18:33.319ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2457 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = f9d8d687-c155-45bc-9271-6c51e4e2bf1d response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
9312025-03-04T03:18:33.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = bb70c531-e01b-4003-8860-f5a50925c56b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
9322025-03-04T03:18:33.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = bb70c531-e01b-4003-8860-f5a50925c56b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
9332025-03-04T03:18:33.322ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3322 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = bb70c531-e01b-4003-8860-f5a50925c56b response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
9342025-03-04T03:18:33.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6256a1c-49a2-4aaf-8d4b-4fb9445bc8d0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9352025-03-04T03:18:33.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6256a1c-49a2-4aaf-8d4b-4fb9445bc8d0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9362025-03-04T03:18:33.741ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2936 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6256a1c-49a2-4aaf-8d4b-4fb9445bc8d0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9372025-03-04T03:18:34.759ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a09b4ba0-ecf3-4c2c-a44e-9b6b76db40b8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9382025-03-04T03:18:34.759ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a09b4ba0-ecf3-4c2c-a44e-9b6b76db40b8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9392025-03-04T03:18:34.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 14057 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a09b4ba0-ecf3-4c2c-a44e-9b6b76db40b8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9402025-03-04T03:18:35.218ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9412025-03-04T03:18:35.219ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9422025-03-04T03:18:35.220ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9432025-03-04T03:18:35.220ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9442025-03-04T03:18:35.222ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9452025-03-04T03:18:35.224ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
9462025-03-04T03:18:35.224ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
9472025-03-04T03:18:35.791ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 094cbf5e-0165-4da3-928e-be2c67a55d31 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9482025-03-04T03:18:35.792ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 094cbf5e-0165-4da3-928e-be2c67a55d31 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9492025-03-04T03:18:35.794ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2526 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 094cbf5e-0165-4da3-928e-be2c67a55d31 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9502025-03-04T03:18:36.814ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b74aec9-1d60-432b-90f6-42b0e10871d5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9512025-03-04T03:18:36.814ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b74aec9-1d60-432b-90f6-42b0e10871d5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9522025-03-04T03:18:36.816ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2181 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b74aec9-1d60-432b-90f6-42b0e10871d5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
9532025-03-04T03:18:37.201ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
9542025-03-04T03:18:37.201ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9552025-03-04T03:18:37.201ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
9562025-03-04T03:18:37.201ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9572025-03-04T03:18:37.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9582025-03-04T03:18:37.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9592025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
9602025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
9612025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9622025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9632025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Timeout
9642025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9652025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 2 reason = Timeout
9662025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9672025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
9682025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
9692025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
9702025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9712025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9722025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
9732025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9742025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9752025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9762025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9772025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9782025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9792025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9802025-03-04T03:18:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9812025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
9822025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
9832025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 4 reason = Timeout
9842025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9852025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9862025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
9872025-03-04T03:18:37.203ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
9882025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9892025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
9902025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9912025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9922025-03-04T03:18:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9932025-03-04T03:18:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9942025-03-04T03:18:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9952025-03-04T03:18:37.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9962025-03-04T03:18:37.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9972025-03-04T03:18:37.205ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.950986ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
9982025-03-04T03:18:37.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9992025-03-04T03:18:37.206ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 2 reason = Timeout
10002025-03-04T03:18:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
10012025-03-04T03:18:37.206ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
10022025-03-04T03:18:37.206ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10032025-03-04T03:18:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
10042025-03-04T03:18:37.206ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10052025-03-04T03:18:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
10062025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10072025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
10082025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10092025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
10102025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10112025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10122025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10132025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10142025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10152025-03-04T03:18:37.207ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10162025-03-04T03:18:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10172025-03-04T03:18:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10182025-03-04T03:18:37.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10192025-03-04T03:18:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10202025-03-04T03:18:37.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10212025-03-04T03:18:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10222025-03-04T03:18:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10232025-03-04T03:18:37.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10242025-03-04T03:18:37.210ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10252025-03-04T03:18:37.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10262025-03-04T03:18:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10272025-03-04T03:18:37.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10282025-03-04T03:18:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10292025-03-04T03:18:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10302025-03-04T03:18:37.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10312025-03-04T03:18:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10322025-03-04T03:18:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10332025-03-04T03:18:37.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10342025-03-04T03:18:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10352025-03-04T03:18:37.218ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10362025-03-04T03:18:37.218ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
10372025-03-04T03:18:37.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10382025-03-04T03:18:37.218ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10392025-03-04T03:18:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10402025-03-04T03:18:37.218ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1029, tv_nsec: 417524931 } })
10412025-03-04T03:18:37.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10422025-03-04T03:18:37.219ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
10432025-03-04T03:18:37.223ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10442025-03-04T03:18:37.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10452025-03-04T03:18:37.225ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:80 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
10462025-03-04T03:18:37.225ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 23.346466ms iteration = 4 status = {"error":"rack not yet initialized: accfd3a5-958c-4905-8c5c-5f2ac43dda05"}
10472025-03-04T03:18:37.225ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10482025-03-04T03:18:37.226ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10492025-03-04T03:18:37.226ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 19.496859ms iteration = 2 status = {"errors":[],"requests_created_ok":[]}
10502025-03-04T03:18:37.227ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 24.946006ms iteration = 2 status = {}
10512025-03-04T03:18:37.227ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10522025-03-04T03:18:37.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10532025-03-04T03:18:37.228ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10542025-03-04T03:18:37.228ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10552025-03-04T03:18:37.239ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
10562025-03-04T03:18:37.239ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 36.893462ms iteration = 4 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
10572025-03-04T03:18:37.249ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10582025-03-04T03:18:37.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10592025-03-04T03:18:37.252ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
10602025-03-04T03:18:37.252ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 48.368313ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
10612025-03-04T03:18:37.253ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 51.016815ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10622025-03-04T03:18:37.261ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.204535ms iteration = 2 status = {"errors":[],"finish_invoked_ok":[]}
10632025-03-04T03:18:37.264ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10642025-03-04T03:18:37.265ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
10652025-03-04T03:18:37.265ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10662025-03-04T03:18:37.265ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a617610-fa0a-44ec-a24c-92ab72a1128d", "content-length": "26", "date": "Tue, 04 Mar 2025 03:18:37 GMT"} })
10672025-03-04T03:18:37.265ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10682025-03-04T03:18:37.267ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10692025-03-04T03:18:37.267ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10702025-03-04T03:18:37.267ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10712025-03-04T03:18:37.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10722025-03-04T03:18:37.270ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10732025-03-04T03:18:37.270ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10742025-03-04T03:18:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10752025-03-04T03:18:37.272ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10762025-03-04T03:18:37.272ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10772025-03-04T03:18:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10782025-03-04T03:18:37.273ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 66.585966ms iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
10792025-03-04T03:18:37.273ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10802025-03-04T03:18:37.273ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
10812025-03-04T03:18:37.273ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10822025-03-04T03:18:37.273ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 66.97273ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10832025-03-04T03:18:37.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10842025-03-04T03:18:37.274ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10852025-03-04T03:18:37.276ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 72.358717ms iteration = 4 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10862025-03-04T03:18:37.276ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10872025-03-04T03:18:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10882025-03-04T03:18:37.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10892025-03-04T03:18:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10902025-03-04T03:18:37.279ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
10912025-03-04T03:18:37.279ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, time_created: 2025-03-04T03:18:08.381559Z, time_modified: 2025-03-04T03:18:08.381559Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:770
10922025-03-04T03:18:37.291ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10932025-03-04T03:18:37.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10942025-03-04T03:18:37.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10952025-03-04T03:18:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10962025-03-04T03:18:37.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10972025-03-04T03:18:37.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
10982025-03-04T03:18:37.297ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10992025-03-04T03:18:37.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11002025-03-04T03:18:37.299ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11012025-03-04T03:18:37.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11022025-03-04T03:18:37.305ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 104.120156ms iteration = 2 status = {}
11032025-03-04T03:18:37.308ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
11042025-03-04T03:18:37.308ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 106.654227ms iteration = 2 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
11052025-03-04T03:18:37.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11062025-03-04T03:18:37.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11072025-03-04T03:18:37.319ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1029, tv_nsec: 518528049 } })
11082025-03-04T03:18:37.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11092025-03-04T03:18:37.320ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
11102025-03-04T03:18:37.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11112025-03-04T03:18:37.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 115.182492ms iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
11122025-03-04T03:18:37.327ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 8dbd4845-bcf6-46ff-9ea5-96e86a6151ef roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
11132025-03-04T03:18:37.327ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 8dbd4845-bcf6-46ff-9ea5-96e86a6151ef resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
11142025-03-04T03:18:37.329ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11152025-03-04T03:18:37.329ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
11162025-03-04T03:18:37.329ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 125.796482ms iteration = 2 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
11172025-03-04T03:18:37.330ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3291 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 8dbd4845-bcf6-46ff-9ea5-96e86a6151ef response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
11182025-03-04T03:18:37.341ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11192025-03-04T03:18:37.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11202025-03-04T03:18:37.360ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
11212025-03-04T03:18:37.360ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11222025-03-04T03:18:37.360ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf59b162-613f-4ec3-a01a-5daf8198523f", "content-length": "26", "date": "Tue, 04 Mar 2025 03:18:37 GMT"} })
11232025-03-04T03:18:37.360ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11242025-03-04T03:18:37.361ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11252025-03-04T03:18:37.368ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11262025-03-04T03:18:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11272025-03-04T03:18:37.375ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11282025-03-04T03:18:37.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11292025-03-04T03:18:37.402ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
11302025-03-04T03:18:37.402ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11312025-03-04T03:18:37.403ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4e4ab7a-b138-4bc1-996b-f53285e1616b", "content-length": "312", "date": "Tue, 04 Mar 2025 03:18:37 GMT"} })
11322025-03-04T03:18:37.403ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11332025-03-04T03:18:37.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11342025-03-04T03:18:37.413ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11352025-03-04T03:18:37.414ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "09c6b53f-ffc3-494d-8f6a-8b78abdeef91", "content-length": "1", "date": "Tue, 04 Mar 2025 03:18:36 GMT"} })
11362025-03-04T03:18:37.415ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11372025-03-04T03:18:37.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11382025-03-04T03:18:37.417ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 216.311262ms iteration = 2 status = null
11392025-03-04T03:18:37.437ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11402025-03-04T03:18:37.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11412025-03-04T03:18:37.528ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11422025-03-04T03:18:37.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11432025-03-04T03:18:37.572ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11442025-03-04T03:18:37.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11452025-03-04T03:18:37.581ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1029, tv_nsec: 779878632 } })
11462025-03-04T03:18:37.581ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1155
11472025-03-04T03:18:37.612ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
11482025-03-04T03:18:37.612ZDEBGnexus (MgsClient): client request background_task = service_zone_nat_tracker body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11492025-03-04T03:18:37.613ZDEBGnexus (MgsClient): client response background_task = service_zone_nat_tracker result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5588ad37-b00d-4a7a-8c9c-60acd39da079", "content-length": "26", "date": "Tue, 04 Mar 2025 03:18:37 GMT"} })
11502025-03-04T03:18:37.613ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11512025-03-04T03:18:37.613ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11522025-03-04T03:18:37.644ZDEBGnexus (DpdClient): client request background_task = service_zone_nat_tracker body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
11532025-03-04T03:18:37.644ZDEBGnexus (DpdClient): client response background_task = service_zone_nat_tracker result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "c609323f-1fc7-4141-9e58-400fd7299efb", "content-length": "4", "date": "Tue, 04 Mar 2025 03:18:36 GMT"} })
11542025-03-04T03:18:37.645ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 443.511445ms iteration = 2 status = 6
11552025-03-04T03:18:37.834ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1b32e3e-b805-4ae4-8c35-35b32458fa6f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
11562025-03-04T03:18:37.835ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1b32e3e-b805-4ae4-8c35-35b32458fa6f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
11572025-03-04T03:18:37.837ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2620 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1b32e3e-b805-4ae4-8c35-35b32458fa6f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
11582025-03-04T03:18:38.029ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11592025-03-04T03:18:38.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11602025-03-04T03:18:38.030ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11612025-03-04T03:18:38.033ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11622025-03-04T03:18:38.033ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11632025-03-04T03:18:38.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11642025-03-04T03:18:38.035ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11652025-03-04T03:18:38.036ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11662025-03-04T03:18:38.036ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11672025-03-04T03:18:38.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11682025-03-04T03:18:38.062ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
11692025-03-04T03:18:38.062ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
11702025-03-04T03:18:38.062ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e087cdf8-6c95-4c06-92c0-067cd3959e12", "date": "Tue, 04 Mar 2025 03:18:37 GMT"} })
11712025-03-04T03:18:38.062ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11722025-03-04T03:18:38.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11732025-03-04T03:18:38.069ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
11742025-03-04T03:18:38.077ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3d1c50e-e25f-44ae-9dad-99061b3d8836", "content-length": "691", "date": "Tue, 04 Mar 2025 03:18:37 GMT"} })
11752025-03-04T03:18:38.077ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
11762025-03-04T03:18:38.077ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
11772025-03-04T03:18:38.077ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
11782025-03-04T03:18:38.079ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 876.869276ms iteration = 2 status = {}
11792025-03-04T03:18:38.160ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11802025-03-04T03:18:38.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11812025-03-04T03:18:38.166ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11822025-03-04T03:18:38.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11832025-03-04T03:18:38.173ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11842025-03-04T03:18:38.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11852025-03-04T03:18:38.175ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11862025-03-04T03:18:38.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11872025-03-04T03:18:38.178ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11882025-03-04T03:18:38.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11892025-03-04T03:18:38.180ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11902025-03-04T03:18:38.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11912025-03-04T03:18:38.182ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11922025-03-04T03:18:38.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11932025-03-04T03:18:38.183ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11942025-03-04T03:18:38.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11952025-03-04T03:18:38.185ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11962025-03-04T03:18:38.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11972025-03-04T03:18:38.186ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11982025-03-04T03:18:38.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
11992025-03-04T03:18:38.190ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12002025-03-04T03:18:38.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12012025-03-04T03:18:38.191ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12022025-03-04T03:18:38.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12032025-03-04T03:18:38.193ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12042025-03-04T03:18:38.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12052025-03-04T03:18:38.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12062025-03-04T03:18:38.198ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
12072025-03-04T03:18:38.198ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12082025-03-04T03:18:38.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12092025-03-04T03:18:38.210ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12102025-03-04T03:18:38.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12112025-03-04T03:18:38.212ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12122025-03-04T03:18:38.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12132025-03-04T03:18:38.214ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12142025-03-04T03:18:38.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12152025-03-04T03:18:38.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12162025-03-04T03:18:38.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12172025-03-04T03:18:38.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12182025-03-04T03:18:38.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12192025-03-04T03:18:38.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12202025-03-04T03:18:38.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12212025-03-04T03:18:38.220ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12222025-03-04T03:18:38.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12232025-03-04T03:18:38.222ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12242025-03-04T03:18:38.227ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ById(001de000-c470-4000-8000-000000000003) } result = Ok(())
12252025-03-04T03:18:38.227ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12262025-03-04T03:18:38.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12272025-03-04T03:18:38.229ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12282025-03-04T03:18:38.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12292025-03-04T03:18:38.232ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12302025-03-04T03:18:38.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12312025-03-04T03:18:38.233ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12322025-03-04T03:18:38.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12332025-03-04T03:18:38.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12342025-03-04T03:18:38.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12352025-03-04T03:18:38.237ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12362025-03-04T03:18:38.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12372025-03-04T03:18:38.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12382025-03-04T03:18:38.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12392025-03-04T03:18:38.239ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12402025-03-04T03:18:38.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ById(001de000-c470-4000-8000-000000000002) } result = Ok(())
12412025-03-04T03:18:38.243ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12422025-03-04T03:18:38.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12432025-03-04T03:18:38.245ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12442025-03-04T03:18:38.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12452025-03-04T03:18:38.246ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12462025-03-04T03:18:38.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12472025-03-04T03:18:38.248ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12482025-03-04T03:18:38.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12492025-03-04T03:18:38.249ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12502025-03-04T03:18:38.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12512025-03-04T03:18:38.251ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12522025-03-04T03:18:38.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12532025-03-04T03:18:38.252ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12542025-03-04T03:18:38.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12552025-03-04T03:18:38.254ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12562025-03-04T03:18:38.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ById(001de000-c470-4000-8000-000000000001) } result = Ok(())
12572025-03-04T03:18:38.257ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12582025-03-04T03:18:38.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12592025-03-04T03:18:38.259ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12602025-03-04T03:18:38.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12612025-03-04T03:18:38.261ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12622025-03-04T03:18:38.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12632025-03-04T03:18:38.262ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12642025-03-04T03:18:38.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12652025-03-04T03:18:38.264ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12662025-03-04T03:18:38.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12672025-03-04T03:18:38.268ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12682025-03-04T03:18:38.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12692025-03-04T03:18:38.269ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12702025-03-04T03:18:38.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12712025-03-04T03:18:38.271ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12722025-03-04T03:18:38.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
12732025-03-04T03:18:38.274ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12742025-03-04T03:18:38.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12752025-03-04T03:18:38.276ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12762025-03-04T03:18:38.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12772025-03-04T03:18:38.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12782025-03-04T03:18:38.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12792025-03-04T03:18:38.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12802025-03-04T03:18:38.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12812025-03-04T03:18:38.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12822025-03-04T03:18:38.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12832025-03-04T03:18:38.283ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12842025-03-04T03:18:38.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12852025-03-04T03:18:38.284ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12862025-03-04T03:18:38.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12872025-03-04T03:18:38.285ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12882025-03-04T03:18:38.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ByName("boundary-ntp") } result = Ok(())
12892025-03-04T03:18:38.289ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12902025-03-04T03:18:38.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12912025-03-04T03:18:38.290ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12922025-03-04T03:18:38.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12932025-03-04T03:18:38.292ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12942025-03-04T03:18:38.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12952025-03-04T03:18:38.293ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12962025-03-04T03:18:38.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12972025-03-04T03:18:38.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12982025-03-04T03:18:38.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12992025-03-04T03:18:38.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13002025-03-04T03:18:38.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13012025-03-04T03:18:38.298ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13022025-03-04T03:18:38.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13032025-03-04T03:18:38.299ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13042025-03-04T03:18:38.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
13052025-03-04T03:18:38.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13062025-03-04T03:18:38.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13072025-03-04T03:18:38.307ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13082025-03-04T03:18:38.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13092025-03-04T03:18:38.308ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13102025-03-04T03:18:38.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13112025-03-04T03:18:38.310ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13122025-03-04T03:18:38.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13132025-03-04T03:18:38.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = cfa519dd-b881-4fe9-9200-b36799f3eae7 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
13142025-03-04T03:18:38.332ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13152025-03-04T03:18:38.332ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = cfa519dd-b881-4fe9-9200-b36799f3eae7 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
13162025-03-04T03:18:38.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13172025-03-04T03:18:38.334ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13182025-03-04T03:18:38.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13192025-03-04T03:18:38.335ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13202025-03-04T03:18:38.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13212025-03-04T03:18:38.337ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13222025-03-04T03:18:38.338ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 6210 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = cfa519dd-b881-4fe9-9200-b36799f3eae7 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
13232025-03-04T03:18:38.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 093c64f5-e9a5-4096-aacd-d5bd1ea15f37 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
13242025-03-04T03:18:38.341ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 093c64f5-e9a5-4096-aacd-d5bd1ea15f37 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
13252025-03-04T03:18:38.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000002, lookup_type: ByName("default") } result = Ok(())
13262025-03-04T03:18:38.343ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2793 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41988 req_id = 093c64f5-e9a5-4096-aacd-d5bd1ea15f37 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
13272025-03-04T03:18:38.348ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13282025-03-04T03:18:38.350ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "23d676fb-45f3-4b33-908d-0f3484deba4c", "date": "Tue, 04 Mar 2025 03:18:37 GMT"} })
13292025-03-04T03:18:38.351ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 1.147861188s iteration = 2 status = {}
13302025-03-04T03:18:38.855ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6aa33adc-fa18-4e3d-a419-f44d6f2b1652 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
13312025-03-04T03:18:38.856ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6aa33adc-fa18-4e3d-a419-f44d6f2b1652 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
13322025-03-04T03:18:38.858ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2997 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6aa33adc-fa18-4e3d-a419-f44d6f2b1652 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
13332025-03-04T03:18:39.280ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
13342025-03-04T03:18:39.280ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
13352025-03-04T03:18:39.281ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
13362025-03-04T03:18:39.282ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
13372025-03-04T03:18:39.283ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13382025-03-04T03:18:39.285ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: accfd3a5-958c-4905-8c5c-5f2ac43dda05, lookup_type: ById(accfd3a5-958c-4905-8c5c-5f2ac43dda05) } result = Ok(())
13392025-03-04T03:18:39.285ZINFOnexus: Rack initialized file = nexus/src/app/rack.rs:767
13402025-03-04T03:18:39.285ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13412025-03-04T03:18:39.285ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13422025-03-04T03:18:39.293ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13432025-03-04T03:18:39.293ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13442025-03-04T03:18:39.298ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13452025-03-04T03:18:39.298ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13462025-03-04T03:18:39.303ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13472025-03-04T03:18:39.304ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13482025-03-04T03:18:39.305ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13492025-03-04T03:18:39.306ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13502025-03-04T03:18:39.307ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13512025-03-04T03:18:39.307ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13522025-03-04T03:18:39.311ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13532025-03-04T03:18:39.314ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13542025-03-04T03:18:39.314ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13552025-03-04T03:18:39.315ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13562025-03-04T03:18:39.316ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13572025-03-04T03:18:39.317ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13582025-03-04T03:18:39.319ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13592025-03-04T03:18:39.319ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13602025-03-04T03:18:39.321ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13612025-03-04T03:18:39.327ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13622025-03-04T03:18:39.327ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13632025-03-04T03:18:39.327ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13642025-03-04T03:18:39.333ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13652025-03-04T03:18:39.333ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13662025-03-04T03:18:39.335ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13672025-03-04T03:18:39.335ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13682025-03-04T03:18:39.337ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13692025-03-04T03:18:39.338ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13702025-03-04T03:18:39.339ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13712025-03-04T03:18:39.339ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13722025-03-04T03:18:39.341ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13732025-03-04T03:18:39.341ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13742025-03-04T03:18:39.342ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13752025-03-04T03:18:39.342ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13762025-03-04T03:18:39.344ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13772025-03-04T03:18:39.347ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13782025-03-04T03:18:39.352ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13792025-03-04T03:18:39.352ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13802025-03-04T03:18:39.353ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13812025-03-04T03:18:39.354ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13822025-03-04T03:18:39.361ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13832025-03-04T03:18:39.361ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13842025-03-04T03:18:39.363ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13852025-03-04T03:18:39.363ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13862025-03-04T03:18:39.365ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13872025-03-04T03:18:39.365ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13882025-03-04T03:18:39.366ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13892025-03-04T03:18:39.367ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13902025-03-04T03:18:39.368ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13912025-03-04T03:18:39.368ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13922025-03-04T03:18:39.370ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13932025-03-04T03:18:39.376ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13942025-03-04T03:18:39.377ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13952025-03-04T03:18:39.377ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13962025-03-04T03:18:39.378ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13972025-03-04T03:18:39.378ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13982025-03-04T03:18:39.380ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
13992025-03-04T03:18:39.380ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14002025-03-04T03:18:39.382ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14012025-03-04T03:18:39.385ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
14022025-03-04T03:18:39.385ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14032025-03-04T03:18:39.385ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14042025-03-04T03:18:39.393ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14052025-03-04T03:18:39.393ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14062025-03-04T03:18:39.395ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14072025-03-04T03:18:39.395ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14082025-03-04T03:18:39.397ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14092025-03-04T03:18:39.397ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14102025-03-04T03:18:39.407ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14112025-03-04T03:18:39.407ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14122025-03-04T03:18:39.410ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14132025-03-04T03:18:39.410ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14142025-03-04T03:18:39.412ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14152025-03-04T03:18:39.412ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14162025-03-04T03:18:39.414ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14172025-03-04T03:18:39.415ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14182025-03-04T03:18:39.420ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14192025-03-04T03:18:39.425ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14202025-03-04T03:18:39.425ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14212025-03-04T03:18:39.425ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14222025-03-04T03:18:39.426ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14232025-03-04T03:18:39.426ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14242025-03-04T03:18:39.428ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14252025-03-04T03:18:39.428ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14262025-03-04T03:18:39.430ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14272025-03-04T03:18:39.434ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14282025-03-04T03:18:39.434ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14292025-03-04T03:18:39.434ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14302025-03-04T03:18:39.446ZDEBGnexus (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 1efc0800-e2a1-43fb-8159-014fa2b2d648, kind: Service { id: 72700cb3-c938-46f0-9b9c-46cf389d975e }, name: Name("external-dns-72700cb3-c938-46f0-9b9c-46cf389d975e"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 246])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6aab6111-ac14-4d7e-808e-18a2dbaec1a3, kind: Service { id: 0e2dad65-6466-48b5-bc3a-8a9f5178c9b7 }, name: Name("external-dns-0e2dad65-6466-48b5-bc3a-8a9f5178c9b7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 05a86b80-7d75-4c2d-8253-0cdbdce17c69, kind: Service { id: 2afd3cd1-228b-4949-95e6-cb3413cc5ec2 }, name: Name("nexus-2afd3cd1-228b-4949-95e6-cb3413cc5ec2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 61])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3ce82d4d-d64c-418d-9d43-fc9e659efafa, kind: Service { id: c28b34ab-1b39-4325-87ad-c4ccf5b009d5 }, name: Name("nexus-c28b34ab-1b39-4325-87ad-c4ccf5b009d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 509b968a-6934-4b81-be1c-676fe22e2948, kind: Service { id: d8484452-7d43-42ac-a588-575b6d31b882 }, name: Name("nexus-d8484452-7d43-42ac-a588-575b6d31b882"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 93])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
14312025-03-04T03:18:39.446ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14322025-03-04T03:18:39.446ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14332025-03-04T03:18:39.447ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14342025-03-04T03:18:39.449ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
14352025-03-04T03:18:39.449ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14362025-03-04T03:18:39.449ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14372025-03-04T03:18:39.451ZDEBGnexus (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
14382025-03-04T03:18:39.460ZDEBGnexus (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1efc0800-e2a1-43fb-8159-014fa2b2d648, kind: Service { id: 72700cb3-c938-46f0-9b9c-46cf389d975e }, name: Name("external-dns-72700cb3-c938-46f0-9b9c-46cf389d975e"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 246])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6aab6111-ac14-4d7e-808e-18a2dbaec1a3, kind: Service { id: 0e2dad65-6466-48b5-bc3a-8a9f5178c9b7 }, name: Name("external-dns-0e2dad65-6466-48b5-bc3a-8a9f5178c9b7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 05a86b80-7d75-4c2d-8253-0cdbdce17c69, kind: Service { id: 2afd3cd1-228b-4949-95e6-cb3413cc5ec2 }, name: Name("nexus-2afd3cd1-228b-4949-95e6-cb3413cc5ec2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 61])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3ce82d4d-d64c-418d-9d43-fc9e659efafa, kind: Service { id: c28b34ab-1b39-4325-87ad-c4ccf5b009d5 }, name: Name("nexus-c28b34ab-1b39-4325-87ad-c4ccf5b009d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 509b968a-6934-4b81-be1c-676fe22e2948, kind: Service { id: d8484452-7d43-42ac-a588-575b6d31b882 }, name: Name("nexus-d8484452-7d43-42ac-a588-575b6d31b882"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 93])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
14392025-03-04T03:18:39.460ZDEBGnexus (InternalApi): resolved 2 rules for sleds
14402025-03-04T03:18:39.565ZDEBGnexus (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: 5525e426-7299-43ab-ad2b-346968b24e1c, time_created: 2025-03-04T03:18:30.785539Z, time_modified: 2025-03-04T03:18:30.785539Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
14412025-03-04T03:18:39.565ZDEBGnexus (InternalApi): sending firewall rules to sled agents
14422025-03-04T03:18:39.565ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14432025-03-04T03:18:39.565ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14442025-03-04T03:18:39.568ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
14452025-03-04T03:18:39.568ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14462025-03-04T03:18:39.570ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14472025-03-04T03:18:39.574ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
14482025-03-04T03:18:39.616ZDEBGnexus (InternalApi): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14492025-03-04T03:18:39.617ZDEBGnexus (InternalApi): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "46a3482c-e707-49da-a71f-1af05740ff0c", "date": "Tue, 04 Mar 2025 03:18:39 GMT"} })
14502025-03-04T03:18:39.618ZINFOnexus (InternalApi): updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:496
14512025-03-04T03:18:39.618ZINFOnexus: plumbed initial IP allowlist file = nexus/src/app/allow_list.rs:161
14522025-03-04T03:18:39.877ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ee497e15-7cbb-4448-8cec-15c24e2fd2b7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14532025-03-04T03:18:39.877ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ee497e15-7cbb-4448-8cec-15c24e2fd2b7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14542025-03-04T03:18:39.879ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1965 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ee497e15-7cbb-4448-8cec-15c24e2fd2b7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14552025-03-04T03:18:40.900ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e889528-8c72-47b6-8234-280719132c90 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14562025-03-04T03:18:40.900ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e889528-8c72-47b6-8234-280719132c90 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14572025-03-04T03:18:40.903ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2777 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e889528-8c72-47b6-8234-280719132c90 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14582025-03-04T03:18:41.924ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eff90991-68d3-4f7f-8d81-93ff180be39f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14592025-03-04T03:18:41.924ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eff90991-68d3-4f7f-8d81-93ff180be39f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14602025-03-04T03:18:41.926ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2010 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eff90991-68d3-4f7f-8d81-93ff180be39f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14612025-03-04T03:18:42.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 632bfb88-296e-4de5-b259-7b3ea60e9a68 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14622025-03-04T03:18:42.946ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 632bfb88-296e-4de5-b259-7b3ea60e9a68 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14632025-03-04T03:18:42.948ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2329 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 632bfb88-296e-4de5-b259-7b3ea60e9a68 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14642025-03-04T03:18:43.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9a8b1e2d-7f7e-4dfe-8c00-1d5ad00e68db roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14652025-03-04T03:18:43.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9a8b1e2d-7f7e-4dfe-8c00-1d5ad00e68db resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14662025-03-04T03:18:43.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2270 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9a8b1e2d-7f7e-4dfe-8c00-1d5ad00e68db response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14672025-03-04T03:18:44.991ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1071906b-5e2e-47a4-b1e5-5d93eb8b39c5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14682025-03-04T03:18:44.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1071906b-5e2e-47a4-b1e5-5d93eb8b39c5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14692025-03-04T03:18:44.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1533 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1071906b-5e2e-47a4-b1e5-5d93eb8b39c5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14702025-03-04T03:18:46.012ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84fb6556-825a-444c-9168-3e3f6610a0c0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14712025-03-04T03:18:46.013ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84fb6556-825a-444c-9168-3e3f6610a0c0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14722025-03-04T03:18:46.014ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2211 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84fb6556-825a-444c-9168-3e3f6610a0c0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14732025-03-04T03:18:47.034ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 20c62158-15b1-4ef8-a25c-22f48ebffac7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14742025-03-04T03:18:47.034ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 20c62158-15b1-4ef8-a25c-22f48ebffac7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14752025-03-04T03:18:47.035ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1976 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 20c62158-15b1-4ef8-a25c-22f48ebffac7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
14762025-03-04T03:18:47.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
14772025-03-04T03:18:47.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14782025-03-04T03:18:47.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14792025-03-04T03:18:47.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 5 reason = Timeout
14802025-03-04T03:18:47.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14812025-03-04T03:18:47.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14822025-03-04T03:18:47.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14832025-03-04T03:18:47.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14842025-03-04T03:18:47.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14852025-03-04T03:18:47.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14862025-03-04T03:18:47.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14872025-03-04T03:18:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14882025-03-04T03:18:47.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14892025-03-04T03:18:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
14902025-03-04T03:18:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14912025-03-04T03:18:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14922025-03-04T03:18:47.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14932025-03-04T03:18:47.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14942025-03-04T03:18:47.269ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.915418ms iteration = 5 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14952025-03-04T03:18:47.271ZINFOnexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/tasks/blueprint_load.rs:88 target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
14962025-03-04T03:18:47.271ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 69.431351ms iteration = 5 status = {"enabled":false,"status":"first target blueprint","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z","time_found":"2025-03-04T03:18:47.271487957Z"}
14972025-03-04T03:18:47.271ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
14982025-03-04T03:18:47.271ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
14992025-03-04T03:18:47.271ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
15002025-03-04T03:18:47.271ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.33\u{b5}s iteration = 2 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
15012025-03-04T03:18:47.271ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15022025-03-04T03:18:47.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15032025-03-04T03:18:47.271ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15042025-03-04T03:18:47.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15052025-03-04T03:18:47.272ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15062025-03-04T03:18:47.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15072025-03-04T03:18:47.272ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15082025-03-04T03:18:47.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15092025-03-04T03:18:47.272ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15102025-03-04T03:18:47.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15112025-03-04T03:18:47.274ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15122025-03-04T03:18:47.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
15132025-03-04T03:18:47.276ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15142025-03-04T03:18:47.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15152025-03-04T03:18:47.276ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15162025-03-04T03:18:47.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
15172025-03-04T03:18:47.277ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15182025-03-04T03:18:47.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15192025-03-04T03:18:47.277ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15202025-03-04T03:18:47.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
15212025-03-04T03:18:47.279ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15222025-03-04T03:18:47.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15232025-03-04T03:18:47.279ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15242025-03-04T03:18:47.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
15252025-03-04T03:18:47.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15262025-03-04T03:18:47.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15272025-03-04T03:18:47.281ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15282025-03-04T03:18:47.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
15292025-03-04T03:18:47.282ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15302025-03-04T03:18:47.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15312025-03-04T03:18:47.286ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 14.543108ms iteration = 2 status = {"nsuccess":5}
15322025-03-04T03:18:48.056ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f6edbeb-b896-4049-8a61-df68b0b8d20b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15332025-03-04T03:18:48.056ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f6edbeb-b896-4049-8a61-df68b0b8d20b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15342025-03-04T03:18:48.058ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1879 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f6edbeb-b896-4049-8a61-df68b0b8d20b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15352025-03-04T03:18:49.077ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c9118485-ec64-4fa7-b1c6-638d17651354 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15362025-03-04T03:18:49.077ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c9118485-ec64-4fa7-b1c6-638d17651354 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15372025-03-04T03:18:49.079ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2034 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c9118485-ec64-4fa7-b1c6-638d17651354 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15382025-03-04T03:18:50.098ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eead666a-4555-46de-915c-3eb70848d9ad roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15392025-03-04T03:18:50.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eead666a-4555-46de-915c-3eb70848d9ad resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15402025-03-04T03:18:50.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2618 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eead666a-4555-46de-915c-3eb70848d9ad response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15412025-03-04T03:18:51.119ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 060b17ff-122a-40f7-8241-067e92dd07da roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15422025-03-04T03:18:51.119ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 060b17ff-122a-40f7-8241-067e92dd07da resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15432025-03-04T03:18:51.121ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2296 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 060b17ff-122a-40f7-8241-067e92dd07da response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15442025-03-04T03:18:52.139ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca353159-58da-4650-b1e6-62ccc413fe8f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15452025-03-04T03:18:52.140ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca353159-58da-4650-b1e6-62ccc413fe8f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15462025-03-04T03:18:52.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2963 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca353159-58da-4650-b1e6-62ccc413fe8f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15472025-03-04T03:18:53.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1915ddd5-ba5d-4bb9-858b-f4035c8ed9c7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15482025-03-04T03:18:53.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1915ddd5-ba5d-4bb9-858b-f4035c8ed9c7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15492025-03-04T03:18:53.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2351 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1915ddd5-ba5d-4bb9-858b-f4035c8ed9c7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15502025-03-04T03:18:54.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 078349a7-95b3-4da1-9ffb-21a667332ea0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15512025-03-04T03:18:54.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 078349a7-95b3-4da1-9ffb-21a667332ea0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15522025-03-04T03:18:54.183ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2205 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 078349a7-95b3-4da1-9ffb-21a667332ea0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15532025-03-04T03:18:55.201ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0d0c01a0-cb6d-4131-98e7-55c5620b383a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15542025-03-04T03:18:55.201ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0d0c01a0-cb6d-4131-98e7-55c5620b383a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15552025-03-04T03:18:55.203ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2553 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0d0c01a0-cb6d-4131-98e7-55c5620b383a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15562025-03-04T03:18:56.222ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32720ecc-0267-4b30-a3c8-ee32c55bfbb7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15572025-03-04T03:18:56.222ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32720ecc-0267-4b30-a3c8-ee32c55bfbb7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15582025-03-04T03:18:56.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1816 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32720ecc-0267-4b30-a3c8-ee32c55bfbb7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15592025-03-04T03:18:57.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
15602025-03-04T03:18:57.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15612025-03-04T03:18:57.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15622025-03-04T03:18:57.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 6 reason = Timeout
15632025-03-04T03:18:57.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15642025-03-04T03:18:57.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15652025-03-04T03:18:57.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15662025-03-04T03:18:57.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15672025-03-04T03:18:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15682025-03-04T03:18:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15692025-03-04T03:18:57.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15702025-03-04T03:18:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15712025-03-04T03:18:57.207ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.32797ms iteration = 6 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15722025-03-04T03:18:57.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15732025-03-04T03:18:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15742025-03-04T03:18:57.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15752025-03-04T03:18:57.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
15762025-03-04T03:18:57.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15772025-03-04T03:18:57.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15782025-03-04T03:18:57.243ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 722d8d56-41ef-4c3e-aa5b-19230772f5ab roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15792025-03-04T03:18:57.243ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 722d8d56-41ef-4c3e-aa5b-19230772f5ab resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15802025-03-04T03:18:57.245ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2115 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 722d8d56-41ef-4c3e-aa5b-19230772f5ab response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15812025-03-04T03:18:57.382ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
15822025-03-04T03:18:57.383ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 181.002788ms iteration = 6 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
15832025-03-04T03:18:58.264ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5a804aba-eb84-4e3f-9333-dc3da4e69023 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15842025-03-04T03:18:58.264ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5a804aba-eb84-4e3f-9333-dc3da4e69023 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15852025-03-04T03:18:58.265ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1587 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5a804aba-eb84-4e3f-9333-dc3da4e69023 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15862025-03-04T03:18:59.285ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3f195d70-22d7-494a-ad8f-bea98d55f6c6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15872025-03-04T03:18:59.285ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3f195d70-22d7-494a-ad8f-bea98d55f6c6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15882025-03-04T03:18:59.287ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2453 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3f195d70-22d7-494a-ad8f-bea98d55f6c6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15892025-03-04T03:19:00.305ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1a07aee-6d3c-4650-996f-f82d2eb0ee6d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15902025-03-04T03:19:00.306ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1a07aee-6d3c-4650-996f-f82d2eb0ee6d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15912025-03-04T03:19:00.307ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1924 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1a07aee-6d3c-4650-996f-f82d2eb0ee6d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15922025-03-04T03:19:01.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5b54b3ac-5edb-4f25-b397-a31a9e21e5ad roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15932025-03-04T03:19:01.327ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5b54b3ac-5edb-4f25-b397-a31a9e21e5ad resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15942025-03-04T03:19:01.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2556 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5b54b3ac-5edb-4f25-b397-a31a9e21e5ad response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15952025-03-04T03:19:02.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0947068f-3ccc-4007-b3bc-5f1f59b88a3f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15962025-03-04T03:19:02.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0947068f-3ccc-4007-b3bc-5f1f59b88a3f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15972025-03-04T03:19:02.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2434 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0947068f-3ccc-4007-b3bc-5f1f59b88a3f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15982025-03-04T03:19:03.368ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ad28b2c5-89e3-4684-a10e-520a498a1ad4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
15992025-03-04T03:19:03.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ad28b2c5-89e3-4684-a10e-520a498a1ad4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16002025-03-04T03:19:03.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2407 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ad28b2c5-89e3-4684-a10e-520a498a1ad4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16012025-03-04T03:19:04.391ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8ca7bfc-6901-4b77-83a4-b36892377d70 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16022025-03-04T03:19:04.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8ca7bfc-6901-4b77-83a4-b36892377d70 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16032025-03-04T03:19:04.393ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1986 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8ca7bfc-6901-4b77-83a4-b36892377d70 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16042025-03-04T03:19:05.412ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e2b9bfdd-becd-47a4-be36-2fa0844b01f3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16052025-03-04T03:19:05.412ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e2b9bfdd-becd-47a4-be36-2fa0844b01f3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16062025-03-04T03:19:05.414ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2320 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e2b9bfdd-becd-47a4-be36-2fa0844b01f3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16072025-03-04T03:19:06.434ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a16840e3-e63c-489c-9d48-d34b46bf6f1f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16082025-03-04T03:19:06.434ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a16840e3-e63c-489c-9d48-d34b46bf6f1f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16092025-03-04T03:19:06.436ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2003 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a16840e3-e63c-489c-9d48-d34b46bf6f1f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
16102025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
16112025-03-04T03:19:07.201ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
16122025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 20.301\u{b5}s iteration = 3 status = {"error":"no config"}
16132025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
16142025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
16152025-03-04T03:19:07.201ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:09:07.201580305 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
16162025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
16172025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
16182025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
16192025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 3 reason = Timeout
16202025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
16212025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16222025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
16232025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
16242025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
16252025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16262025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
16272025-03-04T03:19:07.201ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
16282025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 6.21\u{b5}s iteration = 3 status = {"error":"no config"}
16292025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16302025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
16312025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
16322025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16332025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16342025-03-04T03:19:07.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16352025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
16362025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16372025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
16382025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
16392025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16402025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16412025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
16422025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
16432025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16442025-03-04T03:19:07.202ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
16452025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16462025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
16472025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16482025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
16492025-03-04T03:19:07.202ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
16502025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.43\u{b5}s iteration = 3 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
16512025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16522025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
16532025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
16542025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
16552025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16562025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16572025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
16582025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 3 reason = Timeout
16592025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16602025-03-04T03:19:07.202ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1059, tv_nsec: 400769518 } })
16612025-03-04T03:19:07.202ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1059, tv_nsec: 400792639 } })
16622025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16632025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16642025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16652025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16662025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
16672025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16682025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16692025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16702025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
16712025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.059853ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
16722025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
16732025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.178294ms iteration = 2 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
16742025-03-04T03:19:07.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16752025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16762025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
16772025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
16782025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16792025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
16802025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
16812025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
16822025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
16832025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16842025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16852025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
16862025-03-04T03:19:07.203ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
16872025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16882025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16892025-03-04T03:19:07.203ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1059, tv_nsec: 402015594 } })
16902025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
16912025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16922025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16932025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16942025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16952025-03-04T03:19:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16962025-03-04T03:19:07.203ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
16972025-03-04T03:19:07.204ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:09:07.201580305 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
16982025-03-04T03:19:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.026547ms iteration = 2 status = {"expiration":"2025-03-04T03:09:07.201580305Z","pruned":[]}
16992025-03-04T03:19:07.205ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
17002025-03-04T03:19:07.205ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
17012025-03-04T03:19:07.205ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17022025-03-04T03:19:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
17032025-03-04T03:19:07.205ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1059, tv_nsec: 403886296 } })
17042025-03-04T03:19:07.205ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
17052025-03-04T03:19:07.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17062025-03-04T03:19:07.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17072025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17082025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17092025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17102025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 3 reason = Timeout
17112025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
17122025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
17132025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
17142025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
17152025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
17162025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
17172025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
17182025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
17192025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17202025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
17212025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
17222025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
17232025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.557758ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
17242025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
17252025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
17262025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
17272025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
17282025-03-04T03:19:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17292025-03-04T03:19:07.208ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17302025-03-04T03:19:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17312025-03-04T03:19:07.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17322025-03-04T03:19:07.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17332025-03-04T03:19:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17342025-03-04T03:19:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
17352025-03-04T03:19:07.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17362025-03-04T03:19:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17372025-03-04T03:19:07.210ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17382025-03-04T03:19:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
17392025-03-04T03:19:07.211ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
17402025-03-04T03:19:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17412025-03-04T03:19:07.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17422025-03-04T03:19:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
17432025-03-04T03:19:07.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17442025-03-04T03:19:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17452025-03-04T03:19:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
17462025-03-04T03:19:07.212ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
17472025-03-04T03:19:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17482025-03-04T03:19:07.213ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17492025-03-04T03:19:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17502025-03-04T03:19:07.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17512025-03-04T03:19:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17522025-03-04T03:19:07.214ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17532025-03-04T03:19:07.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17542025-03-04T03:19:07.216ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17552025-03-04T03:19:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
17562025-03-04T03:19:07.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17572025-03-04T03:19:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17582025-03-04T03:19:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17592025-03-04T03:19:07.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17602025-03-04T03:19:07.218ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17612025-03-04T03:19:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17622025-03-04T03:19:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17632025-03-04T03:19:07.220ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17642025-03-04T03:19:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17652025-03-04T03:19:07.227ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.153836ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
17662025-03-04T03:19:07.228ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1059, tv_nsec: 427384684 } })
17672025-03-04T03:19:07.229ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
17682025-03-04T03:19:07.229ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17692025-03-04T03:19:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17702025-03-04T03:19:07.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17712025-03-04T03:19:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17722025-03-04T03:19:07.232ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17732025-03-04T03:19:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17742025-03-04T03:19:07.252ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17752025-03-04T03:19:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17762025-03-04T03:19:07.255ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17772025-03-04T03:19:07.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17782025-03-04T03:19:07.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17792025-03-04T03:19:07.261ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
17802025-03-04T03:19:07.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17812025-03-04T03:19:07.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17822025-03-04T03:19:07.262ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 55.040224ms iteration = 3 status = {"errors":[],"finish_invoked_ok":[]}
17832025-03-04T03:19:07.263ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17842025-03-04T03:19:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17852025-03-04T03:19:07.267ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17862025-03-04T03:19:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17872025-03-04T03:19:07.268ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
17882025-03-04T03:19:07.268ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
17892025-03-04T03:19:07.268ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 63.219915ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
17902025-03-04T03:19:07.268ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.391362ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17912025-03-04T03:19:07.269ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
17922025-03-04T03:19:07.269ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
17932025-03-04T03:19:07.269ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
17942025-03-04T03:19:07.269ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17952025-03-04T03:19:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17962025-03-04T03:19:07.270ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
17972025-03-04T03:19:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17982025-03-04T03:19:07.271ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17992025-03-04T03:19:07.271ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18002025-03-04T03:19:07.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.25837ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
18012025-03-04T03:19:07.272ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18022025-03-04T03:19:07.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18032025-03-04T03:19:07.273ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18042025-03-04T03:19:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18052025-03-04T03:19:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
18062025-03-04T03:19:07.275ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18072025-03-04T03:19:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18082025-03-04T03:19:07.275ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
18092025-03-04T03:19:07.275ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
18102025-03-04T03:19:07.275ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 73.866635ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
18112025-03-04T03:19:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18122025-03-04T03:19:07.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18132025-03-04T03:19:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18142025-03-04T03:19:07.276ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18152025-03-04T03:19:07.276ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
18162025-03-04T03:19:07.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18172025-03-04T03:19:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18182025-03-04T03:19:07.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18192025-03-04T03:19:07.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18202025-03-04T03:19:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18212025-03-04T03:19:07.278ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18222025-03-04T03:19:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18232025-03-04T03:19:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
18242025-03-04T03:19:07.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18252025-03-04T03:19:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18262025-03-04T03:19:07.283ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18272025-03-04T03:19:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
18282025-03-04T03:19:07.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18292025-03-04T03:19:07.284ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18302025-03-04T03:19:07.285ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18312025-03-04T03:19:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18322025-03-04T03:19:07.286ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 78.772756ms iteration = 3 status = {"errors":[],"requests_created_ok":[]}
18332025-03-04T03:19:07.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18342025-03-04T03:19:07.286ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18352025-03-04T03:19:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18362025-03-04T03:19:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
18372025-03-04T03:19:07.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18382025-03-04T03:19:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18392025-03-04T03:19:07.287ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
18402025-03-04T03:19:07.287ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 84.572317ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
18412025-03-04T03:19:07.289ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
18422025-03-04T03:19:07.289ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 87.353791ms iteration = 7 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
18432025-03-04T03:19:07.291ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
18442025-03-04T03:19:07.291ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18452025-03-04T03:19:07.291ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a27321dc-8a2c-4020-a897-70ebbae13fd2", "content-length": "26", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
18462025-03-04T03:19:07.291ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18472025-03-04T03:19:07.293ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18482025-03-04T03:19:07.295ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 93.159101ms iteration = 3 status = 0
18492025-03-04T03:19:07.303ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
18502025-03-04T03:19:07.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
18512025-03-04T03:19:07.307ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 104.961276ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
18522025-03-04T03:19:07.309ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
18532025-03-04T03:19:07.309ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18542025-03-04T03:19:07.309ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c18a9b85-0162-4273-8f9a-825fc4d62e40", "content-length": "26", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
18552025-03-04T03:19:07.309ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18562025-03-04T03:19:07.310ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18572025-03-04T03:19:07.317ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18582025-03-04T03:19:07.317ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
18592025-03-04T03:19:07.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18602025-03-04T03:19:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18612025-03-04T03:19:07.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 116.6418ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
18622025-03-04T03:19:07.331ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
18632025-03-04T03:19:07.331ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 129.433986ms iteration = 3 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
18642025-03-04T03:19:07.378ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
18652025-03-04T03:19:07.379ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
18662025-03-04T03:19:07.379ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
18672025-03-04T03:19:07.379ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18682025-03-04T03:19:07.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18692025-03-04T03:19:07.379ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18702025-03-04T03:19:07.379ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
18712025-03-04T03:19:07.379ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 176.664046ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
18722025-03-04T03:19:07.379ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
18732025-03-04T03:19:07.379ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18742025-03-04T03:19:07.380ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0669123-dc08-4543-8c6d-f067492c8129", "content-length": "26", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
18752025-03-04T03:19:07.380ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18762025-03-04T03:19:07.381ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
18772025-03-04T03:19:07.385ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18782025-03-04T03:19:07.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18792025-03-04T03:19:07.386ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
18802025-03-04T03:19:07.386ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
18812025-03-04T03:19:07.386ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18822025-03-04T03:19:07.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18832025-03-04T03:19:07.389ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
18842025-03-04T03:19:07.389ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 186.131342ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
18852025-03-04T03:19:07.399ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 197.4417ms iteration = 5 status = {"physical_disks_added":0}
18862025-03-04T03:19:07.403ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18872025-03-04T03:19:07.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18882025-03-04T03:19:07.414ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 212.579266ms iteration = 3 status = {}
18892025-03-04T03:19:07.429ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 222.174443ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
18902025-03-04T03:19:07.434ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
18912025-03-04T03:19:07.434ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b11d482-5139-42f8-876c-a4b744e1890b", "content-length": "1", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
18922025-03-04T03:19:07.435ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18932025-03-04T03:19:07.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18942025-03-04T03:19:07.455ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
18952025-03-04T03:19:07.455ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
18962025-03-04T03:19:07.455ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b979eb25-ffbc-40b1-81c8-f79f64518c25", "content-length": "372", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
18972025-03-04T03:19:07.455ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18982025-03-04T03:19:07.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a02d375e-f81a-4ab7-8cf9-4f8fac39451b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
18992025-03-04T03:19:07.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19002025-03-04T03:19:07.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a02d375e-f81a-4ab7-8cf9-4f8fac39451b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
19012025-03-04T03:19:07.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19022025-03-04T03:19:07.457ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19032025-03-04T03:19:07.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
19042025-03-04T03:19:07.459ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19052025-03-04T03:19:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19062025-03-04T03:19:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19072025-03-04T03:19:07.459ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19082025-03-04T03:19:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19092025-03-04T03:19:07.460ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19102025-03-04T03:19:07.461ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19112025-03-04T03:19:07.462ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 260.801167ms iteration = 3 status = null
19122025-03-04T03:19:07.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19132025-03-04T03:19:07.462ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19142025-03-04T03:19:07.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19152025-03-04T03:19:07.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19162025-03-04T03:19:07.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19172025-03-04T03:19:07.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19182025-03-04T03:19:07.464ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 8516 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a02d375e-f81a-4ab7-8cf9-4f8fac39451b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
19192025-03-04T03:19:07.464ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19202025-03-04T03:19:07.465ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
19212025-03-04T03:19:07.465ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
19222025-03-04T03:19:07.465ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19232025-03-04T03:19:07.465ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
19242025-03-04T03:19:07.465ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19252025-03-04T03:19:07.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19262025-03-04T03:19:07.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19272025-03-04T03:19:07.466ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19282025-03-04T03:19:07.469ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19292025-03-04T03:19:07.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19302025-03-04T03:19:07.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19312025-03-04T03:19:07.472ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
19322025-03-04T03:19:07.474ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
19332025-03-04T03:19:07.474ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
19342025-03-04T03:19:07.474ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 273.137027ms iteration = 2 status = {"generation":2}
19352025-03-04T03:19:07.474ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
19362025-03-04T03:19:07.480ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
19372025-03-04T03:19:07.480ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
19382025-03-04T03:19:07.480ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
19392025-03-04T03:19:07.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 278.586764ms iteration = 2 status = {"generation":1}
19402025-03-04T03:19:07.480ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
19412025-03-04T03:19:07.539ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
19422025-03-04T03:19:07.539ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 338.142264ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
19432025-03-04T03:19:07.585ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
19442025-03-04T03:19:07.586ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5f9fb85-6056-4a7b-9505-2b3597e292a8", "content-length": "691", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
19452025-03-04T03:19:07.586ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
19462025-03-04T03:19:07.586ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
19472025-03-04T03:19:07.586ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
19482025-03-04T03:19:07.587ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 385.616535ms iteration = 3 status = {}
19492025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = / versions = all
19502025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /access versions = all
19512025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /assets/{path} versions = all
19522025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/auth versions = all
19532025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/confirm versions = all
19542025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/success versions = all
19552025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/token versions = all
19562025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/verify versions = all
19572025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes versions = all
19582025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /experimental/v1/probes versions = all
19592025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
19602025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes/{probe} versions = all
19612025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles versions = all
19622025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /experimental/v1/system/support-bundles versions = all
19632025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
19642025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
19652025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
19662025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
19672025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
19682025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
19692025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
19702025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /images versions = all
19712025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login versions = all
19722025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/local versions = all
19732025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
19742025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
19752025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
19762025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /lookup/{path} versions = all
19772025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects/{path} versions = all
19782025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects-new versions = all
19792025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /settings/{path} versions = all
19802025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /system/{path} versions = all
19812025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /utilization versions = all
19822025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups versions = all
19832025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/affinity-groups versions = all
19842025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
19852025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
19862025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
19872025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
19882025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
19892025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
19902025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
19912025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups versions = all
19922025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/anti-affinity-groups versions = all
19932025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
19942025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
19952025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
19962025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
19972025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
19982025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
19992025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
20002025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates versions = all
20012025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/certificates versions = all
20022025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/certificates/{certificate} versions = all
20032025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates/{certificate} versions = all
20042025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks versions = all
20052025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks versions = all
20062025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/disks/{disk} versions = all
20072025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks/{disk} versions = all
20082025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
20092025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
20102025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
20112025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/finalize versions = all
20122025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
20132025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips versions = all
20142025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips versions = all
20152025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
20162025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
20172025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
20182025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
20192025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
20202025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups versions = all
20212025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups/{group_id} versions = all
20222025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images versions = all
20232025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images versions = all
20242025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/images/{image} versions = all
20252025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images/{image} versions = all
20262025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/demote versions = all
20272025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/promote versions = all
20282025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances versions = all
20292025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances versions = all
20302025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance} versions = all
20312025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance} versions = all
20322025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/instances/{instance} versions = all
20332025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/disks versions = all
20342025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all
20352025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all
20362025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
20372025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
20382025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
20392025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/reboot versions = all
20402025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
20412025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
20422025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
20432025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/start versions = all
20442025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/stop versions = all
20452025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
20462025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
20472025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
20482025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
20492025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
20502025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
20512025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateways versions = all
20522025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateways versions = all
20532025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
20542025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
20552025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools versions = all
20562025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools/{pool} versions = all
20572025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/login/{silo_name}/local versions = all
20582025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/logout versions = all
20592025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me versions = all
20602025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/groups versions = all
20612025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys versions = all
20622025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/me/ssh-keys versions = all
20632025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
20642025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
20652025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/metrics/{metric_name} versions = all
20662025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces versions = all
20672025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/network-interfaces versions = all
20682025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
20692025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces/{interface} versions = all
20702025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
20712025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ping versions = all
20722025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/policy versions = all
20732025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/policy versions = all
20742025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects versions = all
20752025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/projects versions = all
20762025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/projects/{project} versions = all
20772025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project} versions = all
20782025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project} versions = all
20792025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project}/policy versions = all
20802025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project}/policy versions = all
20812025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots versions = all
20822025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/snapshots versions = all
20832025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
20842025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots/{snapshot} versions = all
20852025-03-04T03:19:07.604ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks versions = all
20862025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
20872025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
20882025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks versions = all
20892025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
20902025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds versions = all
20912025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/sleds versions = all
20922025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
20932025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
20942025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
20952025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
20962025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
20972025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port versions = all
20982025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
20992025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
21002025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
21012025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
21022025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
21032025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches versions = all
21042025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
21052025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers versions = all
21062025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users versions = all
21072025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
21082025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
21092025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/saml versions = all
21102025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
21112025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools versions = all
21122025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools versions = all
21132025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
21142025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
21152025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
21162025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
21172025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
21182025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
21192025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
21202025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
21212025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
21222025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
21232025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
21242025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service versions = all
21252025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
21262025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
21272025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
21282025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
21292025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot versions = all
21302025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/address-lot versions = all
21312025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
21322025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
21332025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/allow-list versions = all
21342025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/allow-list versions = all
21352025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-disable versions = all
21362025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-enable versions = all
21372025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bfd-status versions = all
21382025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp versions = all
21392025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp versions = all
21402025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bgp versions = all
21412025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
21422025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
21432025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
21442025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
21452025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-exported versions = all
21462025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
21472025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
21482025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-status versions = all
21492025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/loopback-address versions = all
21502025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/loopback-address versions = all
21512025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
21522025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
21532025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
21542025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
21552025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
21562025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/policy versions = all
21572025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/policy versions = all
21582025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/roles versions = all
21592025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/roles/{role_name} versions = all
21602025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silo-quotas versions = all
21612025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos versions = all
21622025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/silos versions = all
21632025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/silos/{silo} versions = all
21642025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo} versions = all
21652025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
21662025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
21672025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
21682025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
21692025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
21702025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/timeseries/query versions = all
21712025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/timeseries/schemas versions = all
21722025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/update/repository versions = all
21732025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/repository/{system_version} versions = all
21742025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users versions = all
21752025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users/{user_id} versions = all
21762025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin versions = all
21772025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin/{user} versions = all
21782025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos versions = all
21792025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
21802025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/timeseries/query versions = all
21812025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users versions = all
21822025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/utilization versions = all
21832025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-firewall-rules versions = all
21842025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-firewall-rules versions = all
21852025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes versions = all
21862025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-router-routes versions = all
21872025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
21882025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
21892025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
21902025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers versions = all
21912025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-routers versions = all
21922025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
21932025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers/{router} versions = all
21942025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-routers/{router} versions = all
21952025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets versions = all
21962025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-subnets versions = all
21972025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
21982025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
21992025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
22002025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
22012025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs versions = all
22022025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpcs versions = all
22032025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
22042025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs/{vpc} versions = all
22052025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpcs/{vpc} versions = all
22062025-03-04T03:19:07.605ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = 172.30.2.7:443
22072025-03-04T03:19:07.605ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
22082025-03-04T03:19:07.605ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22092025-03-04T03:19:07.605ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.7:443
22102025-03-04T03:19:07.605ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6e236b57-2f62-4c32-af12-7ba15cafd074", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
22112025-03-04T03:19:07.605ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22122025-03-04T03:19:07.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22132025-03-04T03:19:07.607ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22142025-03-04T03:19:07.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22152025-03-04T03:19:07.609ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22162025-03-04T03:19:07.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22172025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = / versions = all
22182025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /access versions = all
22192025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /assets/{path} versions = all
22202025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/auth versions = all
22212025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/confirm versions = all
22222025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/success versions = all
22232025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/token versions = all
22242025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/verify versions = all
22252025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes versions = all
22262025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /experimental/v1/probes versions = all
22272025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
22282025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
22292025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles versions = all
22302025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /experimental/v1/system/support-bundles versions = all
22312025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
22322025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
22332025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
22342025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
22352025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
22362025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
22372025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
22382025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /images versions = all
22392025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login versions = all
22402025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/local versions = all
22412025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
22422025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
22432025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
22442025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /lookup/{path} versions = all
22452025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects/{path} versions = all
22462025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects-new versions = all
22472025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /settings/{path} versions = all
22482025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /system/{path} versions = all
22492025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /utilization versions = all
22502025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups versions = all
22512025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/affinity-groups versions = all
22522025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
22532025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
22542025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
22552025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
22562025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
22572025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
22582025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
22592025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups versions = all
22602025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/anti-affinity-groups versions = all
22612025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
22622025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
22632025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
22642025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
22652025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
22662025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
22672025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
22682025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates versions = all
22692025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/certificates versions = all
22702025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
22712025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates/{certificate} versions = all
22722025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks versions = all
22732025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks versions = all
22742025-03-04T03:19:07.612ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22752025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/disks/{disk} versions = all
22762025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks/{disk} versions = all
22772025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
22782025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
22792025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
22802025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
22812025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
22822025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips versions = all
22832025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips versions = all
22842025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
22852025-03-04T03:19:07.612ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22862025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
22872025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
22882025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
22892025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
22902025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups versions = all
22912025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups/{group_id} versions = all
22922025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images versions = all
22932025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images versions = all
22942025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/images/{image} versions = all
22952025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images/{image} versions = all
22962025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/demote versions = all
22972025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/promote versions = all
22982025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances versions = all
22992025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances versions = all
23002025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance} versions = all
23012025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance} versions = all
23022025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/instances/{instance} versions = all
23032025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
23042025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
23052025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
23062025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
23072025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
23082025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
23092025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
23102025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
23112025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
23122025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
23132025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/start versions = all
23142025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
23152025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
23162025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
23172025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
23182025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
23192025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
23202025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
23212025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateways versions = all
23222025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateways versions = all
23232025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
23242025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
23252025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools versions = all
23262025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
23272025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
23282025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/logout versions = all
23292025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me versions = all
23302025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/groups versions = all
23312025-03-04T03:19:07.612ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys versions = all
23322025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/me/ssh-keys versions = all
23332025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
23342025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
23352025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
23362025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces versions = all
23372025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/network-interfaces versions = all
23382025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
23392025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
23402025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
23412025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ping versions = all
23422025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/policy versions = all
23432025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/policy versions = all
23442025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects versions = all
23452025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/projects versions = all
23462025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/projects/{project} versions = all
23472025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project} versions = all
23482025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project} versions = all
23492025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project}/policy versions = all
23502025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
23512025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots versions = all
23522025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/snapshots versions = all
23532025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
23542025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
23552025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks versions = all
23562025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
23572025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
23582025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks versions = all
23592025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
23602025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds versions = all
23612025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/sleds versions = all
23622025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
23632025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
23642025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
23652025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
23662025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
23672025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
23682025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
23692025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
23702025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
23712025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
23722025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
23732025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches versions = all
23742025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
23752025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers versions = all
23762025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
23772025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
23782025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
23792025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
23802025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
23812025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools versions = all
23822025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools versions = all
23832025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
23842025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
23852025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
23862025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
23872025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
23882025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
23892025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
23902025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
23912025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
23922025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
23932025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
23942025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service versions = all
23952025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
23962025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
23972025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
23982025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
23992025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot versions = all
24002025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/address-lot versions = all
24012025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
24022025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
24032025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/allow-list versions = all
24042025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
24052025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
24062025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
24072025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
24082025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
24092025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp versions = all
24102025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bgp versions = all
24112025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
24122025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
24132025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
24142025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
24152025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
24162025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
24172025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
24182025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
24192025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
24202025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
24212025-03-04T03:19:07.613ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7d6b84f9-ed08-4281-ab2e-b731e72aa7ce", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
24222025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
24232025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
24242025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
24252025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
24262025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
24272025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/policy versions = all
24282025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/policy versions = all
24292025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/roles versions = all
24302025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/roles/{role_name} versions = all
24312025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silo-quotas versions = all
24322025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos versions = all
24332025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/silos versions = all
24342025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
24352025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo} versions = all
24362025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
24372025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
24382025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
24392025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
24402025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
24412025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/timeseries/query versions = all
24422025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
24432025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/update/repository versions = all
24442025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/repository/{system_version} versions = all
24452025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users versions = all
24462025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users/{user_id} versions = all
24472025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin versions = all
24482025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
24492025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos versions = all
24502025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
24512025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/timeseries/query versions = all
24522025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users versions = all
24532025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/utilization versions = all
24542025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
24552025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
24562025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes versions = all
24572025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-router-routes versions = all
24582025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
24592025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
24602025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
24612025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers versions = all
24622025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-routers versions = all
24632025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
24642025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
24652025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
24662025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets versions = all
24672025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-subnets versions = all
24682025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
24692025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
24702025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
24712025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
24722025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs versions = all
24732025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpcs versions = all
24742025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
24752025-03-04T03:19:07.613ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
24762025-03-04T03:19:07.614ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
24772025-03-04T03:19:07.614ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:12228
24782025-03-04T03:19:07.614ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12228
24792025-03-04T03:19:07.614ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 410.243077ms iteration = 3 status = {}
24802025-03-04T03:19:07.620ZDEBGnexus (nexus-producer-server): registered DTrace probes
24812025-03-04T03:19:07.627ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::c]:40647 method = GET path = /{producer_id} versions = all
24822025-03-04T03:19:07.627ZINFOnexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:40647
24832025-03-04T03:19:07.627ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:40647
24842025-03-04T03:19:07.627ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::c]:40647
24852025-03-04T03:19:07.628ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::c]:12221
24862025-03-04T03:19:07.628ZDEBGnexus (nexus-producer-server): starting producer registration task
24872025-03-04T03:19:07.628ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::c]:40647 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
24882025-03-04T03:19:07.628ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
24892025-03-04T03:19:07.628ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
24902025-03-04T03:19:07.665ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24912025-03-04T03:19:07.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24922025-03-04T03:19:07.668ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
24932025-03-04T03:19:07.668ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24942025-03-04T03:19:07.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24952025-03-04T03:19:07.688ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
24962025-03-04T03:19:07.705ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
24972025-03-04T03:19:07.722ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
24982025-03-04T03:19:07.722ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:34516
24992025-03-04T03:19:07.727ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34516 req_id = 76dea665-cccf-490f-889c-7c09e7e104a5 roles = RoleSet { roles: {} } uri = /metrics/producers
25002025-03-04T03:19:07.727ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34516 req_id = 76dea665-cccf-490f-889c-7c09e7e104a5 resource = Database result = Ok(()) uri = /metrics/producers
25012025-03-04T03:19:07.751ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
25022025-03-04T03:19:07.753ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3c4fe9a-198e-4918-919a-84d21a53116f", "content-length": "134", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
25032025-03-04T03:19:07.756ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
25042025-03-04T03:19:07.756ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
25052025-03-04T03:19:07.756ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
25062025-03-04T03:19:07.756ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
25072025-03-04T03:19:07.758ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
25082025-03-04T03:19:07.758ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
25092025-03-04T03:19:07.758ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
25102025-03-04T03:19:07.758ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
25112025-03-04T03:19:07.758ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
25122025-03-04T03:19:07.759ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
25132025-03-04T03:19:07.759ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
25142025-03-04T03:19:07.759ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25152025-03-04T03:19:07.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25162025-03-04T03:19:07.761ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
25172025-03-04T03:19:07.761ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "06bc1b53-1853-45df-93a3-4e7b2e5a60d9", "content-length": "2", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
25182025-03-04T03:19:07.761ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
25192025-03-04T03:19:07.761ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "96ee82ce-0272-48f4-b6ef-cc4b3f550f62", "content-length": "51", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
25202025-03-04T03:19:07.761ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
25212025-03-04T03:19:07.761ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
25222025-03-04T03:19:07.766ZINFOnexus: assigned collector to new producer collector_id = ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a file = nexus/src/app/oximeter.rs:76 producer_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
25232025-03-04T03:19:07.766ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 43450 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34516 req_id = 76dea665-cccf-490f-889c-7c09e7e104a5 response_code = 201 uri = /metrics/producers
25242025-03-04T03:19:07.766ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "76dea665-cccf-490f-889c-7c09e7e104a5", "content-length": "41", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
25252025-03-04T03:19:07.767ZDEBGnexus (producer-registration-task): registered with nexus successfully
25262025-03-04T03:19:07.767ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
25272025-03-04T03:19:07.818ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
25282025-03-04T03:19:07.833ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
25292025-03-04T03:19:07.833ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "84a45442-d059-4d59-8d0d-d4df76144a93", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
25302025-03-04T03:19:07.833ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
25312025-03-04T03:19:07.834ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "75deee17-3827-4388-aa9b-b5502a10ff1e", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
25322025-03-04T03:19:07.834ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
25332025-03-04T03:19:07.835ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
25342025-03-04T03:19:07.840ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 365.49859ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
25352025-03-04T03:19:07.854ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "461ecdba-0d7b-40d0-bf78-62e5653f56a5", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
25362025-03-04T03:19:07.854ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
25372025-03-04T03:19:07.854ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
25382025-03-04T03:19:07.854ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6802c104-9a21-447f-9076-21d6388ea78b", "content-length": "682", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} })
25392025-03-04T03:19:07.857ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25402025-03-04T03:19:07.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25412025-03-04T03:19:07.859ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
25422025-03-04T03:19:07.860ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "0fa9894d-70bc-449d-b26c-ff2775af47af", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25432025-03-04T03:19:07.860ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25442025-03-04T03:19:07.861ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "55cc60bc-2fbc-41be-9dfd-7908a94b9008", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25452025-03-04T03:19:07.861ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25462025-03-04T03:19:07.862ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "31549ec7-1845-4d0b-9f4a-3355ef160464", "date": "Tue, 04 Mar 2025 03:19:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25472025-03-04T03:19:07.862ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25482025-03-04T03:19:07.863ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25492025-03-04T03:19:07.863ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 383.403329ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
25502025-03-04T03:19:07.865ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25512025-03-04T03:19:07.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25522025-03-04T03:19:07.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25532025-03-04T03:19:07.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25542025-03-04T03:19:07.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25552025-03-04T03:19:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25562025-03-04T03:19:07.894ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 692.463926ms iteration = 3 status = {}
25572025-03-04T03:19:08.484ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a62ea21e-9526-4745-b4c0-432d2d3a9f44 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25582025-03-04T03:19:08.484ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a62ea21e-9526-4745-b4c0-432d2d3a9f44 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25592025-03-04T03:19:08.486ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2245 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a62ea21e-9526-4745-b4c0-432d2d3a9f44 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25602025-03-04T03:19:09.512ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 336912ed-2155-4f53-9015-eae32d4be8d6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25612025-03-04T03:19:09.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 336912ed-2155-4f53-9015-eae32d4be8d6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25622025-03-04T03:19:09.514ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1838 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 336912ed-2155-4f53-9015-eae32d4be8d6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25632025-03-04T03:19:10.532ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c2167798-2535-46a6-985f-89cde0e3cf83 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25642025-03-04T03:19:10.533ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c2167798-2535-46a6-985f-89cde0e3cf83 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25652025-03-04T03:19:10.535ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2221 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c2167798-2535-46a6-985f-89cde0e3cf83 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25662025-03-04T03:19:11.554ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e285bafa-5214-43a3-9208-fa66ae86fb62 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25672025-03-04T03:19:11.554ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e285bafa-5214-43a3-9208-fa66ae86fb62 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25682025-03-04T03:19:11.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2587 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e285bafa-5214-43a3-9208-fa66ae86fb62 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25692025-03-04T03:19:12.576ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d7bc02f-2006-4206-ad9d-e65ccf1aa6fc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25702025-03-04T03:19:12.576ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d7bc02f-2006-4206-ad9d-e65ccf1aa6fc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25712025-03-04T03:19:12.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2600 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d7bc02f-2006-4206-ad9d-e65ccf1aa6fc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25722025-03-04T03:19:13.597ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3f9f5244-1b3e-4296-bd26-15e191fbcc6d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25732025-03-04T03:19:13.597ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3f9f5244-1b3e-4296-bd26-15e191fbcc6d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25742025-03-04T03:19:13.599ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2190 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3f9f5244-1b3e-4296-bd26-15e191fbcc6d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25752025-03-04T03:19:14.618ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1ad8ee1-22f2-4b3a-b61c-2ded31a0da3c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25762025-03-04T03:19:14.619ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1ad8ee1-22f2-4b3a-b61c-2ded31a0da3c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25772025-03-04T03:19:14.620ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1871 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1ad8ee1-22f2-4b3a-b61c-2ded31a0da3c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25782025-03-04T03:19:15.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8cb313e7-2c8c-43cf-bbba-dd535b77c3a9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25792025-03-04T03:19:15.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8cb313e7-2c8c-43cf-bbba-dd535b77c3a9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25802025-03-04T03:19:15.641ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1828 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8cb313e7-2c8c-43cf-bbba-dd535b77c3a9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25812025-03-04T03:19:16.661ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 879313cb-a8d5-4d7d-8dc5-95b59d866d53 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25822025-03-04T03:19:16.661ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 879313cb-a8d5-4d7d-8dc5-95b59d866d53 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25832025-03-04T03:19:16.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2683 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 879313cb-a8d5-4d7d-8dc5-95b59d866d53 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
25842025-03-04T03:19:17.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
25852025-03-04T03:19:17.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25862025-03-04T03:19:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25872025-03-04T03:19:17.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
25882025-03-04T03:19:17.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25892025-03-04T03:19:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25902025-03-04T03:19:17.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25912025-03-04T03:19:17.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25922025-03-04T03:19:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25932025-03-04T03:19:17.206ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.299191ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25942025-03-04T03:19:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25952025-03-04T03:19:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25962025-03-04T03:19:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25972025-03-04T03:19:17.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25982025-03-04T03:19:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25992025-03-04T03:19:17.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26002025-03-04T03:19:17.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
26012025-03-04T03:19:17.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26022025-03-04T03:19:17.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26032025-03-04T03:19:17.223ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
26042025-03-04T03:19:17.223ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.366922ms iteration = 8 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
26052025-03-04T03:19:17.682ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e34a3f16-6678-416a-a874-1485e66ebd4c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26062025-03-04T03:19:17.683ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e34a3f16-6678-416a-a874-1485e66ebd4c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26072025-03-04T03:19:17.685ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2188 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e34a3f16-6678-416a-a874-1485e66ebd4c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26082025-03-04T03:19:18.704ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 34594db0-49b9-4040-9d92-278272d99bf9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26092025-03-04T03:19:18.704ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 34594db0-49b9-4040-9d92-278272d99bf9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26102025-03-04T03:19:18.705ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1946 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 34594db0-49b9-4040-9d92-278272d99bf9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26112025-03-04T03:19:19.723ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 99b61b79-b7fa-43c5-a103-0643d4d39f90 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26122025-03-04T03:19:19.724ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 99b61b79-b7fa-43c5-a103-0643d4d39f90 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26132025-03-04T03:19:19.725ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1846 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 99b61b79-b7fa-43c5-a103-0643d4d39f90 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26142025-03-04T03:19:20.744ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c3d01ce3-b1bb-447c-9b49-5502d0b28ebd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26152025-03-04T03:19:20.744ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c3d01ce3-b1bb-447c-9b49-5502d0b28ebd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26162025-03-04T03:19:20.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2714 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c3d01ce3-b1bb-447c-9b49-5502d0b28ebd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26172025-03-04T03:19:21.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 676d09a7-6ba7-4d14-b2bf-ca3e9548c270 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26182025-03-04T03:19:21.771ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 676d09a7-6ba7-4d14-b2bf-ca3e9548c270 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26192025-03-04T03:19:21.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2568 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 676d09a7-6ba7-4d14-b2bf-ca3e9548c270 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26202025-03-04T03:19:22.791ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 52149a61-0169-4443-9917-31ad6a93e7fa roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26212025-03-04T03:19:22.791ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 52149a61-0169-4443-9917-31ad6a93e7fa resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26222025-03-04T03:19:22.793ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2332 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 52149a61-0169-4443-9917-31ad6a93e7fa response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26232025-03-04T03:19:23.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a09da332-7f9b-4ab2-913e-3e6a8f17e7e5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26242025-03-04T03:19:23.812ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a09da332-7f9b-4ab2-913e-3e6a8f17e7e5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26252025-03-04T03:19:23.814ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1909 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a09da332-7f9b-4ab2-913e-3e6a8f17e7e5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26262025-03-04T03:19:24.833ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 07f0281c-c3ee-435f-890d-9a19f44c0d54 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26272025-03-04T03:19:24.833ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 07f0281c-c3ee-435f-890d-9a19f44c0d54 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26282025-03-04T03:19:24.834ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1848 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 07f0281c-c3ee-435f-890d-9a19f44c0d54 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26292025-03-04T03:19:25.854ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f8f3cc98-becd-4e23-b80d-e8c7f83a2b13 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26302025-03-04T03:19:25.854ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f8f3cc98-becd-4e23-b80d-e8c7f83a2b13 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26312025-03-04T03:19:25.856ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1958 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f8f3cc98-becd-4e23-b80d-e8c7f83a2b13 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26322025-03-04T03:19:26.875ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32e8c9f6-9346-40b9-adf8-e5dd13b37bf0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26332025-03-04T03:19:26.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32e8c9f6-9346-40b9-adf8-e5dd13b37bf0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26342025-03-04T03:19:26.878ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2728 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32e8c9f6-9346-40b9-adf8-e5dd13b37bf0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26352025-03-04T03:19:27.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
26362025-03-04T03:19:27.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26372025-03-04T03:19:27.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26382025-03-04T03:19:27.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
26392025-03-04T03:19:27.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26402025-03-04T03:19:27.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26412025-03-04T03:19:27.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26422025-03-04T03:19:27.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26432025-03-04T03:19:27.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26442025-03-04T03:19:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26452025-03-04T03:19:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26462025-03-04T03:19:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26472025-03-04T03:19:27.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26482025-03-04T03:19:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26492025-03-04T03:19:27.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26502025-03-04T03:19:27.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
26512025-03-04T03:19:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26522025-03-04T03:19:27.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26532025-03-04T03:19:27.231ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
26542025-03-04T03:19:27.231ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.554752ms iteration = 9 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
26552025-03-04T03:19:27.240ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.55082ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26562025-03-04T03:19:27.893ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:40647 remote_addr = [fd00:1122:3344:101::d]:64466
26572025-03-04T03:19:27.903ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 10476 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 0942ad54-76ca-4165-8f68-0c9ee5553764 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
26582025-03-04T03:19:27.904ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6db6eab3-f743-4267-ad01-d5b982ca2965 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26592025-03-04T03:19:27.904ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6db6eab3-f743-4267-ad01-d5b982ca2965 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26602025-03-04T03:19:27.905ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1585 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6db6eab3-f743-4267-ad01-d5b982ca2965 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26612025-03-04T03:19:28.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0637dfa6-07fd-4f0a-bd4e-7ffb8367016d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26622025-03-04T03:19:28.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0637dfa6-07fd-4f0a-bd4e-7ffb8367016d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26632025-03-04T03:19:28.930ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4991 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0637dfa6-07fd-4f0a-bd4e-7ffb8367016d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26642025-03-04T03:19:29.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 99b8634f-06af-444e-bd86-2f41f6dab30c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26652025-03-04T03:19:29.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 99b8634f-06af-444e-bd86-2f41f6dab30c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26662025-03-04T03:19:29.952ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1977 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 99b8634f-06af-444e-bd86-2f41f6dab30c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26672025-03-04T03:19:30.969ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd1df6d4-0987-4c9c-9b84-b0415ed50341 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26682025-03-04T03:19:30.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd1df6d4-0987-4c9c-9b84-b0415ed50341 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26692025-03-04T03:19:30.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2091 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd1df6d4-0987-4c9c-9b84-b0415ed50341 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26702025-03-04T03:19:31.988ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 623ebc99-1d4c-44d8-8443-00ff5a88e258 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26712025-03-04T03:19:31.989ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 623ebc99-1d4c-44d8-8443-00ff5a88e258 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26722025-03-04T03:19:31.990ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2027 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 623ebc99-1d4c-44d8-8443-00ff5a88e258 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26732025-03-04T03:19:33.009ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a886915c-5278-4e0a-b9da-ead7e1562bf3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26742025-03-04T03:19:33.009ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a886915c-5278-4e0a-b9da-ead7e1562bf3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26752025-03-04T03:19:33.011ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2060 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a886915c-5278-4e0a-b9da-ead7e1562bf3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26762025-03-04T03:19:34.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0339c91d-bcd7-4e51-8528-8cff626db0af roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26772025-03-04T03:19:34.032ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0339c91d-bcd7-4e51-8528-8cff626db0af resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26782025-03-04T03:19:34.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2103 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0339c91d-bcd7-4e51-8528-8cff626db0af response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26792025-03-04T03:19:35.053ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1a0ef220-cb9d-4369-a40f-41ea906ce97d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26802025-03-04T03:19:35.053ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1a0ef220-cb9d-4369-a40f-41ea906ce97d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26812025-03-04T03:19:35.055ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2267 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1a0ef220-cb9d-4369-a40f-41ea906ce97d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26822025-03-04T03:19:36.075ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5bdb97b2-d69c-4389-bf6d-d726ca2d5b38 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26832025-03-04T03:19:36.076ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5bdb97b2-d69c-4389-bf6d-d726ca2d5b38 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26842025-03-04T03:19:36.077ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2219 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5bdb97b2-d69c-4389-bf6d-d726ca2d5b38 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26852025-03-04T03:19:37.097ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0cddf0b-1a27-49a9-b7a8-ae0312921c0b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26862025-03-04T03:19:37.097ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0cddf0b-1a27-49a9-b7a8-ae0312921c0b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26872025-03-04T03:19:37.099ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2268 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0cddf0b-1a27-49a9-b7a8-ae0312921c0b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
26882025-03-04T03:19:37.201ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 4 reason = Timeout
26892025-03-04T03:19:37.201ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
26902025-03-04T03:19:37.201ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
26912025-03-04T03:19:37.201ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26922025-03-04T03:19:37.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
26932025-03-04T03:19:37.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26942025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
26952025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
26962025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
26972025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26982025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 4 reason = Timeout
26992025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27002025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
27012025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27022025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
27032025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27042025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
27052025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
27062025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27072025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
27082025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27092025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
27102025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27112025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27122025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27132025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27142025-03-04T03:19:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27152025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27162025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
27172025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
27182025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
27192025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
27202025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
27212025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27222025-03-04T03:19:37.203ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
27232025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27242025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27252025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27262025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
27272025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
27282025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27292025-03-04T03:19:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27302025-03-04T03:19:37.204ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1089, tv_nsec: 402514146 } })
27312025-03-04T03:19:37.204ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
27322025-03-04T03:19:37.204ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1089, tv_nsec: 403046022 } })
27332025-03-04T03:19:37.204ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
27342025-03-04T03:19:37.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27352025-03-04T03:19:37.205ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27362025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27372025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 4 reason = Timeout
27382025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
27392025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
27402025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
27412025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
27422025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
27432025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
27442025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27452025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
27462025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
27472025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27482025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
27492025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
27502025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
27512025-03-04T03:19:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
27522025-03-04T03:19:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
27532025-03-04T03:19:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
27542025-03-04T03:19:37.208ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27552025-03-04T03:19:37.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27562025-03-04T03:19:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27572025-03-04T03:19:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
27582025-03-04T03:19:37.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27592025-03-04T03:19:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
27602025-03-04T03:19:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27612025-03-04T03:19:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
27622025-03-04T03:19:37.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27632025-03-04T03:19:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27642025-03-04T03:19:37.210ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27652025-03-04T03:19:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27662025-03-04T03:19:37.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27672025-03-04T03:19:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27682025-03-04T03:19:37.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27692025-03-04T03:19:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
27702025-03-04T03:19:37.212ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27712025-03-04T03:19:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27722025-03-04T03:19:37.212ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27732025-03-04T03:19:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27742025-03-04T03:19:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
27752025-03-04T03:19:37.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27762025-03-04T03:19:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27772025-03-04T03:19:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
27782025-03-04T03:19:37.214ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
27792025-03-04T03:19:37.214ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27802025-03-04T03:19:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27812025-03-04T03:19:37.216ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27822025-03-04T03:19:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27832025-03-04T03:19:37.223ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27842025-03-04T03:19:37.224ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1089, tv_nsec: 422401859 } })
27852025-03-04T03:19:37.224ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
27862025-03-04T03:19:37.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27872025-03-04T03:19:37.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27882025-03-04T03:19:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27892025-03-04T03:19:37.233ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27902025-03-04T03:19:37.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27912025-03-04T03:19:37.233ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.940777ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27922025-03-04T03:19:37.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27932025-03-04T03:19:37.239ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
27942025-03-04T03:19:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27952025-03-04T03:19:37.239ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
27962025-03-04T03:19:37.239ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 36.443456ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
27972025-03-04T03:19:37.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27982025-03-04T03:19:37.246ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 39.370782ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
27992025-03-04T03:19:37.252ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
28002025-03-04T03:19:37.253ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28012025-03-04T03:19:37.253ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
28022025-03-04T03:19:37.253ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28032025-03-04T03:19:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
28042025-03-04T03:19:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28052025-03-04T03:19:37.253ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28062025-03-04T03:19:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28072025-03-04T03:19:37.255ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
28082025-03-04T03:19:37.255ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
28092025-03-04T03:19:37.255ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.843847ms iteration = 10 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
28102025-03-04T03:19:37.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
28112025-03-04T03:19:37.255ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28122025-03-04T03:19:37.255ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
28132025-03-04T03:19:37.255ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28142025-03-04T03:19:37.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28152025-03-04T03:19:37.255ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28162025-03-04T03:19:37.256ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 54.188624ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
28172025-03-04T03:19:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
28182025-03-04T03:19:37.257ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28192025-03-04T03:19:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28202025-03-04T03:19:37.258ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28212025-03-04T03:19:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28222025-03-04T03:19:37.258ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.248639ms iteration = 4 status = {"errors":[],"finish_invoked_ok":[]}
28232025-03-04T03:19:37.261ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28242025-03-04T03:19:37.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28252025-03-04T03:19:37.263ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28262025-03-04T03:19:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
28272025-03-04T03:19:37.265ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28282025-03-04T03:19:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28292025-03-04T03:19:37.266ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
28302025-03-04T03:19:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28312025-03-04T03:19:37.267ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 60.478491ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
28322025-03-04T03:19:37.268ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 65.926528ms iteration = 4 status = 0
28332025-03-04T03:19:37.268ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 61.19314ms iteration = 4 status = {"errors":[],"requests_created_ok":[]}
28342025-03-04T03:19:37.274ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28352025-03-04T03:19:37.275ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28362025-03-04T03:19:37.275ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28372025-03-04T03:19:37.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28382025-03-04T03:19:37.276ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28392025-03-04T03:19:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28402025-03-04T03:19:37.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28412025-03-04T03:19:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28422025-03-04T03:19:37.286ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
28432025-03-04T03:19:37.286ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28442025-03-04T03:19:37.287ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "494aacc8-67f7-4e17-bc59-7cc0649dadcc", "content-length": "26", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
28452025-03-04T03:19:37.287ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28462025-03-04T03:19:37.288ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28472025-03-04T03:19:37.289ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28482025-03-04T03:19:37.289ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
28492025-03-04T03:19:37.289ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 85.65148ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
28502025-03-04T03:19:37.289ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
28512025-03-04T03:19:37.289ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 87.415682ms iteration = 4 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
28522025-03-04T03:19:37.294ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 86.830214ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
28532025-03-04T03:19:37.296ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
28542025-03-04T03:19:37.296ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28552025-03-04T03:19:37.297ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a83eedd-a91c-4406-a2ec-9e60314d73c4", "content-length": "26", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
28562025-03-04T03:19:37.297ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28572025-03-04T03:19:37.298ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28582025-03-04T03:19:37.310ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
28592025-03-04T03:19:37.310ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28602025-03-04T03:19:37.310ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fc1c34b-1d37-4cc3-adcf-b5530d164198", "content-length": "26", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
28612025-03-04T03:19:37.310ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28622025-03-04T03:19:37.311ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
28632025-03-04T03:19:37.323ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 120.798981ms iteration = 6 status = {"physical_disks_added":0}
28642025-03-04T03:19:37.329ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 126.786344ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
28652025-03-04T03:19:37.335ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28662025-03-04T03:19:37.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28672025-03-04T03:19:37.337ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28682025-03-04T03:19:37.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28692025-03-04T03:19:37.339ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28702025-03-04T03:19:37.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28712025-03-04T03:19:37.341ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28722025-03-04T03:19:37.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28732025-03-04T03:19:37.342ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28742025-03-04T03:19:37.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28752025-03-04T03:19:37.461ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
28762025-03-04T03:19:37.462ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28772025-03-04T03:19:37.462ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1aa38ccc-17a0-4dca-bb1f-58e812b9f556", "content-length": "372", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
28782025-03-04T03:19:37.462ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28792025-03-04T03:19:37.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28802025-03-04T03:19:37.479ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
28812025-03-04T03:19:37.479ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
28822025-03-04T03:19:37.480ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b0529d48-3855-4956-b7e6-d298b0f639bf", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
28832025-03-04T03:19:37.480ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28842025-03-04T03:19:37.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28852025-03-04T03:19:37.482ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28862025-03-04T03:19:37.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28872025-03-04T03:19:37.486ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28882025-03-04T03:19:37.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28892025-03-04T03:19:37.487ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 285.664941ms iteration = 4 status = {}
28902025-03-04T03:19:37.490ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
28912025-03-04T03:19:37.490ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28922025-03-04T03:19:37.490ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e7d89cd3-f297-452b-ad12-9d5e4e36c03f", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
28932025-03-04T03:19:37.491ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 287.98937ms iteration = 4 status = {}
28942025-03-04T03:19:37.500ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
28952025-03-04T03:19:37.500ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a89bc8c0-81e4-453a-9f2a-272d88d1b347", "content-length": "1", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
28962025-03-04T03:19:37.501ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28972025-03-04T03:19:37.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28982025-03-04T03:19:37.506ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
28992025-03-04T03:19:37.506ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "55c50719-2983-4ddd-b84b-f57d90ea811a", "content-length": "691", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
29002025-03-04T03:19:37.506ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
29012025-03-04T03:19:37.506ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
29022025-03-04T03:19:37.506ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
29032025-03-04T03:19:37.508ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 306.670738ms iteration = 4 status = null
29042025-03-04T03:19:37.508ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 306.083341ms iteration = 4 status = {}
29052025-03-04T03:19:37.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29062025-03-04T03:19:37.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29072025-03-04T03:19:37.542ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
29082025-03-04T03:19:37.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29092025-03-04T03:19:37.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29102025-03-04T03:19:37.549ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
29112025-03-04T03:19:37.549ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "84387cd5-6657-46d1-a8bf-5f18d6b3a656", "content-length": "134", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
29122025-03-04T03:19:37.549ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
29132025-03-04T03:19:37.549ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
29142025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
29152025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
29162025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
29172025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
29182025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
29192025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
29202025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
29212025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
29222025-03-04T03:19:37.550ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
29232025-03-04T03:19:37.550ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29242025-03-04T03:19:37.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29252025-03-04T03:19:37.560ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
29262025-03-04T03:19:37.560ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e940fe8-c956-4f10-a5c5-c50ccb8a0603", "content-length": "2", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
29272025-03-04T03:19:37.560ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
29282025-03-04T03:19:37.560ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dfb9ff7-a1ff-43aa-a267-90c561abc580", "content-length": "51", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
29292025-03-04T03:19:37.560ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
29302025-03-04T03:19:37.560ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
29312025-03-04T03:19:37.651ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e9859318-32c8-460c-89ee-d968997e0a9d", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
29322025-03-04T03:19:37.651ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
29332025-03-04T03:19:37.651ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
29342025-03-04T03:19:37.651ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0386caeb-5433-422f-b5a7-5bfb886a0f35", "content-length": "682", "date": "Tue, 04 Mar 2025 03:19:37 GMT"} })
29352025-03-04T03:19:37.651ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29362025-03-04T03:19:37.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29372025-03-04T03:19:37.655ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29382025-03-04T03:19:37.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29392025-03-04T03:19:37.659ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29402025-03-04T03:19:37.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29412025-03-04T03:19:37.661ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29422025-03-04T03:19:37.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29432025-03-04T03:19:37.677ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 475.440017ms iteration = 4 status = {}
29442025-03-04T03:19:37.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 524 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 0129646b-4e4d-41dc-9b4d-cf18961527de response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
29452025-03-04T03:19:38.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f79e96db-1722-4b8b-8781-fbb8bb53ed24 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29462025-03-04T03:19:38.119ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f79e96db-1722-4b8b-8781-fbb8bb53ed24 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29472025-03-04T03:19:38.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1947 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f79e96db-1722-4b8b-8781-fbb8bb53ed24 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29482025-03-04T03:19:39.187ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d2dbf3f-9672-45f9-b73f-baad4ac01317 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29492025-03-04T03:19:39.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d2dbf3f-9672-45f9-b73f-baad4ac01317 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29502025-03-04T03:19:39.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2204 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d2dbf3f-9672-45f9-b73f-baad4ac01317 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29512025-03-04T03:19:40.209ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 92f9b1df-885c-4215-b620-994e9cc91c1b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29522025-03-04T03:19:40.209ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 92f9b1df-885c-4215-b620-994e9cc91c1b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29532025-03-04T03:19:40.211ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1992 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 92f9b1df-885c-4215-b620-994e9cc91c1b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29542025-03-04T03:19:41.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a9bacbd6-3f97-4bc8-8139-37738a4774df roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29552025-03-04T03:19:41.231ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a9bacbd6-3f97-4bc8-8139-37738a4774df resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29562025-03-04T03:19:41.233ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2818 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a9bacbd6-3f97-4bc8-8139-37738a4774df response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29572025-03-04T03:19:42.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dd5a3d0a-8997-4b69-a469-acb7e82f8a1d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29582025-03-04T03:19:42.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dd5a3d0a-8997-4b69-a469-acb7e82f8a1d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29592025-03-04T03:19:42.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2436 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dd5a3d0a-8997-4b69-a469-acb7e82f8a1d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29602025-03-04T03:19:43.274ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f540762-1e0f-46ec-839a-bde09e83cb88 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29612025-03-04T03:19:43.275ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f540762-1e0f-46ec-839a-bde09e83cb88 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29622025-03-04T03:19:43.276ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1932 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f540762-1e0f-46ec-839a-bde09e83cb88 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29632025-03-04T03:19:44.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 62aa4045-c3d5-4105-8228-80d0db3725f1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29642025-03-04T03:19:44.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 62aa4045-c3d5-4105-8228-80d0db3725f1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29652025-03-04T03:19:44.297ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2113 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 62aa4045-c3d5-4105-8228-80d0db3725f1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29662025-03-04T03:19:45.316ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84a759f1-c5f6-486a-b9df-45980469a540 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29672025-03-04T03:19:45.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84a759f1-c5f6-486a-b9df-45980469a540 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29682025-03-04T03:19:45.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1952 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84a759f1-c5f6-486a-b9df-45980469a540 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29692025-03-04T03:19:46.336ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f92a8585-5716-458d-a6eb-bf655b768e8d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29702025-03-04T03:19:46.337ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f92a8585-5716-458d-a6eb-bf655b768e8d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29712025-03-04T03:19:46.339ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2530 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f92a8585-5716-458d-a6eb-bf655b768e8d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29722025-03-04T03:19:47.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
29732025-03-04T03:19:47.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29742025-03-04T03:19:47.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29752025-03-04T03:19:47.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
29762025-03-04T03:19:47.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29772025-03-04T03:19:47.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29782025-03-04T03:19:47.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29792025-03-04T03:19:47.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29802025-03-04T03:19:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29812025-03-04T03:19:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29822025-03-04T03:19:47.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29832025-03-04T03:19:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29842025-03-04T03:19:47.206ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.514653ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29852025-03-04T03:19:47.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29862025-03-04T03:19:47.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29872025-03-04T03:19:47.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29882025-03-04T03:19:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
29892025-03-04T03:19:47.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29902025-03-04T03:19:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29912025-03-04T03:19:47.221ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
29922025-03-04T03:19:47.221ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.656668ms iteration = 11 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
29932025-03-04T03:19:47.358ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7566f7b6-6712-4996-a655-ee780811f56a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29942025-03-04T03:19:47.358ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7566f7b6-6712-4996-a655-ee780811f56a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29952025-03-04T03:19:47.360ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2710 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7566f7b6-6712-4996-a655-ee780811f56a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29962025-03-04T03:19:47.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 621 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = cdf99689-9980-4130-8afd-a29f974c2bc7 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
29972025-03-04T03:19:48.381ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = df84d94e-e65e-4ce7-8614-472b5cde3d83 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29982025-03-04T03:19:48.382ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = df84d94e-e65e-4ce7-8614-472b5cde3d83 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
29992025-03-04T03:19:48.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2037 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = df84d94e-e65e-4ce7-8614-472b5cde3d83 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30002025-03-04T03:19:49.403ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 628f3cc4-300d-41f3-b559-236f0424fe0c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30012025-03-04T03:19:49.403ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 628f3cc4-300d-41f3-b559-236f0424fe0c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30022025-03-04T03:19:49.404ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1818 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 628f3cc4-300d-41f3-b559-236f0424fe0c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30032025-03-04T03:19:50.423ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25bc38db-3724-4b6a-8486-bca8c59ccf7b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30042025-03-04T03:19:50.423ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25bc38db-3724-4b6a-8486-bca8c59ccf7b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30052025-03-04T03:19:50.425ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2390 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25bc38db-3724-4b6a-8486-bca8c59ccf7b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30062025-03-04T03:19:51.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 28330a24-54bf-45d6-a0bb-79243b1b7437 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30072025-03-04T03:19:51.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 28330a24-54bf-45d6-a0bb-79243b1b7437 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30082025-03-04T03:19:51.446ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2210 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 28330a24-54bf-45d6-a0bb-79243b1b7437 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30092025-03-04T03:19:52.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb2a9b44-5934-4103-a7bc-25089d175d2b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30102025-03-04T03:19:52.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb2a9b44-5934-4103-a7bc-25089d175d2b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30112025-03-04T03:19:52.468ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2648 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb2a9b44-5934-4103-a7bc-25089d175d2b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30122025-03-04T03:19:53.487ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cf86ae7-acaa-4326-81a9-fe3a84c56c4a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30132025-03-04T03:19:53.487ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cf86ae7-acaa-4326-81a9-fe3a84c56c4a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30142025-03-04T03:19:53.489ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2222 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cf86ae7-acaa-4326-81a9-fe3a84c56c4a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30152025-03-04T03:19:54.508ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aee1960f-1b1e-45f9-8007-38efc503bf8c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30162025-03-04T03:19:54.508ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aee1960f-1b1e-45f9-8007-38efc503bf8c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30172025-03-04T03:19:54.509ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1835 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aee1960f-1b1e-45f9-8007-38efc503bf8c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30182025-03-04T03:19:55.528ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = adcd9a29-442d-4d61-abb6-65fefe2c3cce roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30192025-03-04T03:19:55.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = adcd9a29-442d-4d61-abb6-65fefe2c3cce resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30202025-03-04T03:19:55.530ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2213 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = adcd9a29-442d-4d61-abb6-65fefe2c3cce response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30212025-03-04T03:19:56.550ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a0dcb6b-4fbd-421e-9462-043b4c8dc3da roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30222025-03-04T03:19:56.551ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a0dcb6b-4fbd-421e-9462-043b4c8dc3da resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30232025-03-04T03:19:56.553ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2392 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a0dcb6b-4fbd-421e-9462-043b4c8dc3da response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30242025-03-04T03:19:57.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
30252025-03-04T03:19:57.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
30262025-03-04T03:19:57.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30272025-03-04T03:19:57.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30282025-03-04T03:19:57.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30292025-03-04T03:19:57.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30302025-03-04T03:19:57.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30312025-03-04T03:19:57.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30322025-03-04T03:19:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30332025-03-04T03:19:57.207ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.08123ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30342025-03-04T03:19:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30352025-03-04T03:19:57.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30362025-03-04T03:19:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30372025-03-04T03:19:57.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30382025-03-04T03:19:57.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30392025-03-04T03:19:57.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30402025-03-04T03:19:57.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
30412025-03-04T03:19:57.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30422025-03-04T03:19:57.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30432025-03-04T03:19:57.223ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
30442025-03-04T03:19:57.223ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.074275ms iteration = 12 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
30452025-03-04T03:19:57.572ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ce857faa-d9d8-429f-b34b-dc9d26d6900f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30462025-03-04T03:19:57.572ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ce857faa-d9d8-429f-b34b-dc9d26d6900f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30472025-03-04T03:19:57.574ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1914 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ce857faa-d9d8-429f-b34b-dc9d26d6900f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30482025-03-04T03:19:57.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 896 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = f39d8bae-d2d5-4cac-ab96-08dba6530d1b response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
30492025-03-04T03:19:58.594ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08deda5e-bbfa-4db8-ab2a-aa036cbdcb33 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30502025-03-04T03:19:58.595ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08deda5e-bbfa-4db8-ab2a-aa036cbdcb33 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30512025-03-04T03:19:58.596ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1520 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08deda5e-bbfa-4db8-ab2a-aa036cbdcb33 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30522025-03-04T03:19:59.615ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 57dfa895-52c6-45c8-a4e6-d580fe512ef9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30532025-03-04T03:19:59.615ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 57dfa895-52c6-45c8-a4e6-d580fe512ef9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30542025-03-04T03:19:59.618ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2704 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 57dfa895-52c6-45c8-a4e6-d580fe512ef9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30552025-03-04T03:20:00.637ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7f8e6bf-116a-4a9a-aad1-9cc3e76c7546 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30562025-03-04T03:20:00.637ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7f8e6bf-116a-4a9a-aad1-9cc3e76c7546 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30572025-03-04T03:20:00.639ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2562 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7f8e6bf-116a-4a9a-aad1-9cc3e76c7546 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30582025-03-04T03:20:01.658ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 83090647-87c5-477c-8303-54f1bbedcfe3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30592025-03-04T03:20:01.658ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 83090647-87c5-477c-8303-54f1bbedcfe3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30602025-03-04T03:20:01.660ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2549 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 83090647-87c5-477c-8303-54f1bbedcfe3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30612025-03-04T03:20:02.679ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f0c29a9f-ee68-41a4-880e-66596e96e8bf roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30622025-03-04T03:20:02.679ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f0c29a9f-ee68-41a4-880e-66596e96e8bf resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30632025-03-04T03:20:02.681ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2471 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f0c29a9f-ee68-41a4-880e-66596e96e8bf response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30642025-03-04T03:20:03.699ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 18244e9f-7384-4c07-8a49-46e9898ab780 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30652025-03-04T03:20:03.700ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 18244e9f-7384-4c07-8a49-46e9898ab780 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30662025-03-04T03:20:03.701ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2246 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 18244e9f-7384-4c07-8a49-46e9898ab780 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30672025-03-04T03:20:04.720ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5841e6ed-b11a-4970-ac15-4e9591b2a1a4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30682025-03-04T03:20:04.720ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5841e6ed-b11a-4970-ac15-4e9591b2a1a4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30692025-03-04T03:20:04.722ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2043 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5841e6ed-b11a-4970-ac15-4e9591b2a1a4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30702025-03-04T03:20:05.741ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4216f0a9-8f96-4430-8a53-7a7fcc9e5574 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30712025-03-04T03:20:05.741ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4216f0a9-8f96-4430-8a53-7a7fcc9e5574 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30722025-03-04T03:20:05.743ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2210 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4216f0a9-8f96-4430-8a53-7a7fcc9e5574 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30732025-03-04T03:20:06.763ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 750677d6-d029-4880-9a26-ceddf7d00a98 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30742025-03-04T03:20:06.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 750677d6-d029-4880-9a26-ceddf7d00a98 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30752025-03-04T03:20:06.765ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2628 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 750677d6-d029-4880-9a26-ceddf7d00a98 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
30762025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
30772025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
30782025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
30792025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30802025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30812025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
30822025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
30832025-03-04T03:20:07.201ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:10:07.201747310 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
30842025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
30852025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
30862025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
30872025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
30882025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
30892025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30902025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
30912025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
30922025-03-04T03:20:07.201ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
30932025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30942025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
30952025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30962025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
30972025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30982025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30992025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31002025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
31012025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
31022025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
31032025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31042025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
31052025-03-04T03:20:07.202ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
31062025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31072025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
31082025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.76\u{b5}s iteration = 4 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
31092025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31102025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
31112025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 5 reason = Timeout
31122025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31132025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31142025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
31152025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31162025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
31172025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
31182025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
31192025-03-04T03:20:07.202ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
31202025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
31212025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
31222025-03-04T03:20:07.202ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1119, tv_nsec: 400829967 } })
31232025-03-04T03:20:07.202ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1119, tv_nsec: 400840037 } })
31242025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31252025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31262025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31272025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31282025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31292025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
31302025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
31312025-03-04T03:20:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31322025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
31332025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.273415ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
31342025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
31352025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.410007ms iteration = 3 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
31362025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
31372025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
31382025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31392025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Timeout
31402025-03-04T03:20:07.203ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
31412025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
31422025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
31432025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31442025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
31452025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31462025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
31472025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31482025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
31492025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31502025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31512025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31522025-03-04T03:20:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31532025-03-04T03:20:07.204ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1119, tv_nsec: 402289105 } })
31542025-03-04T03:20:07.204ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
31552025-03-04T03:20:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31562025-03-04T03:20:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31572025-03-04T03:20:07.205ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31582025-03-04T03:20:07.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31592025-03-04T03:20:07.205ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:10:07.201747310 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
31602025-03-04T03:20:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.368721ms iteration = 3 status = {"expiration":"2025-03-04T03:10:07.201747310Z","pruned":[]}
31612025-03-04T03:20:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31622025-03-04T03:20:07.206ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
31632025-03-04T03:20:07.206ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
31642025-03-04T03:20:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
31652025-03-04T03:20:07.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
31662025-03-04T03:20:07.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31672025-03-04T03:20:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31682025-03-04T03:20:07.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1119, tv_nsec: 405404923 } })
31692025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 5 reason = Timeout
31702025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
31712025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
31722025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
31732025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
31742025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
31752025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
31762025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31772025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
31782025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
31792025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
31802025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
31812025-03-04T03:20:07.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
31822025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
31832025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31842025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
31852025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
31862025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
31872025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
31882025-03-04T03:20:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.380656ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
31892025-03-04T03:20:07.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31902025-03-04T03:20:07.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31912025-03-04T03:20:07.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31922025-03-04T03:20:07.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31932025-03-04T03:20:07.209ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31942025-03-04T03:20:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
31952025-03-04T03:20:07.210ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31962025-03-04T03:20:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31972025-03-04T03:20:07.211ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31982025-03-04T03:20:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31992025-03-04T03:20:07.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32002025-03-04T03:20:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
32012025-03-04T03:20:07.212ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32022025-03-04T03:20:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32032025-03-04T03:20:07.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32042025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32052025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32062025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32072025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
32082025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32092025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32102025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
32112025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
32122025-03-04T03:20:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
32132025-03-04T03:20:07.215ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32142025-03-04T03:20:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32152025-03-04T03:20:07.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32162025-03-04T03:20:07.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32172025-03-04T03:20:07.217ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32182025-03-04T03:20:07.218ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32192025-03-04T03:20:07.218ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32202025-03-04T03:20:07.218ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
32212025-03-04T03:20:07.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32222025-03-04T03:20:07.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32232025-03-04T03:20:07.218ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32242025-03-04T03:20:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32252025-03-04T03:20:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32262025-03-04T03:20:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
32272025-03-04T03:20:07.219ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
32282025-03-04T03:20:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
32292025-03-04T03:20:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32302025-03-04T03:20:07.222ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32312025-03-04T03:20:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32322025-03-04T03:20:07.222ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
32332025-03-04T03:20:07.222ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
32342025-03-04T03:20:07.222ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
32352025-03-04T03:20:07.222ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32362025-03-04T03:20:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32372025-03-04T03:20:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32382025-03-04T03:20:07.223ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
32392025-03-04T03:20:07.223ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
32402025-03-04T03:20:07.223ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 16.734615ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
32412025-03-04T03:20:07.224ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32422025-03-04T03:20:07.224ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
32432025-03-04T03:20:07.224ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
32442025-03-04T03:20:07.224ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
32452025-03-04T03:20:07.224ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32462025-03-04T03:20:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32472025-03-04T03:20:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32482025-03-04T03:20:07.225ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 18.192793ms iteration = 5 status = {"errors":[],"finish_invoked_ok":[]}
32492025-03-04T03:20:07.226ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.452389ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
32502025-03-04T03:20:07.227ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 19.917214ms iteration = 5 status = {"errors":[],"requests_created_ok":[]}
32512025-03-04T03:20:07.228ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32522025-03-04T03:20:07.228ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32532025-03-04T03:20:07.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32542025-03-04T03:20:07.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32552025-03-04T03:20:07.228ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1119, tv_nsec: 426881056 } })
32562025-03-04T03:20:07.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32572025-03-04T03:20:07.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32582025-03-04T03:20:07.229ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
32592025-03-04T03:20:07.230ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32602025-03-04T03:20:07.230ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
32612025-03-04T03:20:07.230ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32622025-03-04T03:20:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32632025-03-04T03:20:07.232ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32642025-03-04T03:20:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32652025-03-04T03:20:07.234ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32662025-03-04T03:20:07.237ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32672025-03-04T03:20:07.237ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32682025-03-04T03:20:07.239ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32692025-03-04T03:20:07.239ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32702025-03-04T03:20:07.239ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32712025-03-04T03:20:07.239ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32722025-03-04T03:20:07.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32732025-03-04T03:20:07.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32742025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32752025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32762025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32772025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32782025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32792025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32802025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32812025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32822025-03-04T03:20:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32832025-03-04T03:20:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
32842025-03-04T03:20:07.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32852025-03-04T03:20:07.243ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
32862025-03-04T03:20:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32872025-03-04T03:20:07.243ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 39.896789ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
32882025-03-04T03:20:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
32892025-03-04T03:20:07.243ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32902025-03-04T03:20:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32912025-03-04T03:20:07.249ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32922025-03-04T03:20:07.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32932025-03-04T03:20:07.252ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32942025-03-04T03:20:07.254ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
32952025-03-04T03:20:07.254ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32962025-03-04T03:20:07.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32972025-03-04T03:20:07.264ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
32982025-03-04T03:20:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
32992025-03-04T03:20:07.266ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
33002025-03-04T03:20:07.266ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.197287ms iteration = 13 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
33012025-03-04T03:20:07.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33022025-03-04T03:20:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
33032025-03-04T03:20:07.283ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33042025-03-04T03:20:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33052025-03-04T03:20:07.286ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 79.603366ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
33062025-03-04T03:20:07.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 83.408563ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33072025-03-04T03:20:07.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 80.098182ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
33082025-03-04T03:20:07.288ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33092025-03-04T03:20:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
33102025-03-04T03:20:07.290ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33112025-03-04T03:20:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33122025-03-04T03:20:07.296ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
33132025-03-04T03:20:07.296ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
33142025-03-04T03:20:07.296ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 93.860261ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
33152025-03-04T03:20:07.297ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 95.070455ms iteration = 7 status = {"physical_disks_added":0}
33162025-03-04T03:20:07.299ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 92.507644ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
33172025-03-04T03:20:07.303ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
33182025-03-04T03:20:07.303ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 101.20734ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
33192025-03-04T03:20:07.303ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
33202025-03-04T03:20:07.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
33212025-03-04T03:20:07.304ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
33222025-03-04T03:20:07.304ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
33232025-03-04T03:20:07.304ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 100.963447ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
33242025-03-04T03:20:07.308ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
33252025-03-04T03:20:07.308ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
33262025-03-04T03:20:07.308ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33272025-03-04T03:20:07.308ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
33282025-03-04T03:20:07.308ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
33292025-03-04T03:20:07.308ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33302025-03-04T03:20:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33312025-03-04T03:20:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33322025-03-04T03:20:07.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33332025-03-04T03:20:07.310ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33342025-03-04T03:20:07.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
33352025-03-04T03:20:07.313ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33362025-03-04T03:20:07.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33372025-03-04T03:20:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
33382025-03-04T03:20:07.314ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33392025-03-04T03:20:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33402025-03-04T03:20:07.316ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
33412025-03-04T03:20:07.316ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
33422025-03-04T03:20:07.316ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
33432025-03-04T03:20:07.316ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 114.19805ms iteration = 3 status = {"generation":2}
33442025-03-04T03:20:07.316ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
33452025-03-04T03:20:07.316ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
33462025-03-04T03:20:07.316ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
33472025-03-04T03:20:07.316ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 114.488643ms iteration = 3 status = {"generation":1}
33482025-03-04T03:20:07.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 117.970006ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
33492025-03-04T03:20:07.324ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
33502025-03-04T03:20:07.324ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 121.53556ms iteration = 5 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
33512025-03-04T03:20:07.346ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33522025-03-04T03:20:07.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33532025-03-04T03:20:07.348ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
33542025-03-04T03:20:07.348ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33552025-03-04T03:20:07.348ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c386c68f-e165-4d95-8c26-8d1286a688b2", "content-length": "26", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
33562025-03-04T03:20:07.348ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33572025-03-04T03:20:07.349ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
33582025-03-04T03:20:07.349ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33592025-03-04T03:20:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33602025-03-04T03:20:07.350ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33612025-03-04T03:20:07.362ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
33622025-03-04T03:20:07.362ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33632025-03-04T03:20:07.362ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "335178b4-7fa4-4f93-b4e0-10776cff5fbf", "content-length": "26", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
33642025-03-04T03:20:07.362ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33652025-03-04T03:20:07.364ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33662025-03-04T03:20:07.365ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
33672025-03-04T03:20:07.365ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 161.752593ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
33682025-03-04T03:20:07.366ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
33692025-03-04T03:20:07.393ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
33702025-03-04T03:20:07.393ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33712025-03-04T03:20:07.393ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d2669b4-3238-4380-811b-32ef49917e16", "content-length": "26", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
33722025-03-04T03:20:07.393ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33732025-03-04T03:20:07.395ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
33742025-03-04T03:20:07.495ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
33752025-03-04T03:20:07.495ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33762025-03-04T03:20:07.495ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cfdba83-2b4a-43e4-9551-c6c0423b748b", "content-length": "372", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
33772025-03-04T03:20:07.495ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33782025-03-04T03:20:07.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33792025-03-04T03:20:07.498ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33802025-03-04T03:20:07.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33812025-03-04T03:20:07.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33822025-03-04T03:20:07.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
33832025-03-04T03:20:07.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33842025-03-04T03:20:07.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33852025-03-04T03:20:07.503ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 301.21526ms iteration = 5 status = 0
33862025-03-04T03:20:07.508ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
33872025-03-04T03:20:07.509ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
33882025-03-04T03:20:07.509ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
33892025-03-04T03:20:07.509ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fc92133f-cf61-4a0e-a37f-bf1a8ab85307", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
33902025-03-04T03:20:07.509ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33912025-03-04T03:20:07.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33922025-03-04T03:20:07.511ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33932025-03-04T03:20:07.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33942025-03-04T03:20:07.515ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33952025-03-04T03:20:07.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33962025-03-04T03:20:07.517ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
33972025-03-04T03:20:07.517ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "219af1f8-0a7a-472b-bbf6-d5eaa89affa2", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
33982025-03-04T03:20:07.517ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
33992025-03-04T03:20:07.519ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
34002025-03-04T03:20:07.519ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
34012025-03-04T03:20:07.519ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6ea92b2c-f221-4733-9300-5ab47e18fb2f", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
34022025-03-04T03:20:07.519ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
34032025-03-04T03:20:07.519ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ff1f455c-66f7-44db-a795-316cbba5c49c", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
34042025-03-04T03:20:07.523ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
34052025-03-04T03:20:07.523ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 319.432345ms iteration = 5 status = {}
34062025-03-04T03:20:07.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 321.302587ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
34072025-03-04T03:20:07.526ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 324.542487ms iteration = 5 status = {}
34082025-03-04T03:20:07.527ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
34092025-03-04T03:20:07.528ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "76a9a379-cea8-4528-910c-b970e2fdf373", "content-length": "1", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
34102025-03-04T03:20:07.530ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
34112025-03-04T03:20:07.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
34122025-03-04T03:20:07.532ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 331.150048ms iteration = 5 status = null
34132025-03-04T03:20:07.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34142025-03-04T03:20:07.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34152025-03-04T03:20:07.535ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34162025-03-04T03:20:07.536ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
34172025-03-04T03:20:07.536ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34182025-03-04T03:20:07.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34192025-03-04T03:20:07.538ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34202025-03-04T03:20:07.539ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
34212025-03-04T03:20:07.539ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34222025-03-04T03:20:07.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34232025-03-04T03:20:07.589ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
34242025-03-04T03:20:07.589ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "93353e7a-9cdf-450d-af21-c362c477a1f1", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
34252025-03-04T03:20:07.589ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
34262025-03-04T03:20:07.590ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "0d8d1c53-28e2-4c6d-8fe4-de4ffcbd68a0", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
34272025-03-04T03:20:07.590ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
34282025-03-04T03:20:07.592ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "941b6f94-7574-4baa-ad97-03afbc12afee", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
34292025-03-04T03:20:07.592ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
34302025-03-04T03:20:07.594ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
34312025-03-04T03:20:07.594ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 392.724013ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
34322025-03-04T03:20:07.600ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
34332025-03-04T03:20:07.600ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b398747-b435-4857-84f0-46077cc6b780", "content-length": "691", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
34342025-03-04T03:20:07.600ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
34352025-03-04T03:20:07.600ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
34362025-03-04T03:20:07.600ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
34372025-03-04T03:20:07.602ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 399.908391ms iteration = 5 status = {}
34382025-03-04T03:20:07.627ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34392025-03-04T03:20:07.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34402025-03-04T03:20:07.629ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
34412025-03-04T03:20:07.629ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34422025-03-04T03:20:07.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34432025-03-04T03:20:07.637ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
34442025-03-04T03:20:07.637ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bbca3d2-f934-4709-94b7-7343a1379357", "content-length": "134", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
34452025-03-04T03:20:07.637ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
34462025-03-04T03:20:07.637ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
34472025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
34482025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
34492025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
34502025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
34512025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
34522025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
34532025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
34542025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
34552025-03-04T03:20:07.638ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
34562025-03-04T03:20:07.638ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34572025-03-04T03:20:07.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34582025-03-04T03:20:07.640ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
34592025-03-04T03:20:07.640ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "5de08502-ea05-48d1-823c-14eab5b5ed76", "content-length": "2", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
34602025-03-04T03:20:07.640ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
34612025-03-04T03:20:07.640ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "385f76da-9156-4122-8b61-8b319d0ec75d", "content-length": "51", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
34622025-03-04T03:20:07.640ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
34632025-03-04T03:20:07.640ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
34642025-03-04T03:20:07.733ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2ee5b19d-a2c9-4c2f-80f7-e838fd475768", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
34652025-03-04T03:20:07.733ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
34662025-03-04T03:20:07.733ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
34672025-03-04T03:20:07.733ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "109041b8-4b7b-49fe-a2da-21db2549a676", "content-length": "682", "date": "Tue, 04 Mar 2025 03:20:07 GMT"} })
34682025-03-04T03:20:07.733ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34692025-03-04T03:20:07.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34702025-03-04T03:20:07.736ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34712025-03-04T03:20:07.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34722025-03-04T03:20:07.739ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34732025-03-04T03:20:07.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34742025-03-04T03:20:07.741ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34752025-03-04T03:20:07.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34762025-03-04T03:20:07.747ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 545.251661ms iteration = 5 status = {}
34772025-03-04T03:20:07.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a28f74be-e50e-4af3-b4ed-ffb824b7ced5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34782025-03-04T03:20:07.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a28f74be-e50e-4af3-b4ed-ffb824b7ced5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34792025-03-04T03:20:07.786ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2259 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a28f74be-e50e-4af3-b4ed-ffb824b7ced5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34802025-03-04T03:20:07.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 555 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = f3de51e9-5d4d-4ed6-a4b6-680b3d35f30a response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
34812025-03-04T03:20:08.807ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 811ac93f-63b3-419e-949c-038e3245936b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34822025-03-04T03:20:08.807ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 811ac93f-63b3-419e-949c-038e3245936b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34832025-03-04T03:20:08.809ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2727 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 811ac93f-63b3-419e-949c-038e3245936b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34842025-03-04T03:20:09.831ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d8ead0e-aed5-4070-837e-48df8894b157 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34852025-03-04T03:20:09.831ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d8ead0e-aed5-4070-837e-48df8894b157 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34862025-03-04T03:20:09.832ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1563 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d8ead0e-aed5-4070-837e-48df8894b157 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34872025-03-04T03:20:10.851ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 11782f2b-1113-4b9f-8f7a-4147a34498cb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34882025-03-04T03:20:10.851ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 11782f2b-1113-4b9f-8f7a-4147a34498cb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34892025-03-04T03:20:10.853ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2646 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 11782f2b-1113-4b9f-8f7a-4147a34498cb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34902025-03-04T03:20:11.872ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6b8f0b01-3e62-4f02-b880-1d8267ee2df2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34912025-03-04T03:20:11.873ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6b8f0b01-3e62-4f02-b880-1d8267ee2df2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34922025-03-04T03:20:11.874ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1945 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6b8f0b01-3e62-4f02-b880-1d8267ee2df2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34932025-03-04T03:20:12.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b096589e-8f77-49f5-9ceb-9900945a5961 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34942025-03-04T03:20:12.894ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b096589e-8f77-49f5-9ceb-9900945a5961 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34952025-03-04T03:20:12.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2250 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b096589e-8f77-49f5-9ceb-9900945a5961 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34962025-03-04T03:20:13.915ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8aa55da6-1d48-490e-a8a3-5c252c96008d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34972025-03-04T03:20:13.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8aa55da6-1d48-490e-a8a3-5c252c96008d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34982025-03-04T03:20:13.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2533 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8aa55da6-1d48-490e-a8a3-5c252c96008d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
34992025-03-04T03:20:14.936ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a0358e2-b53f-4486-a899-e3019be9cc16 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35002025-03-04T03:20:14.936ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a0358e2-b53f-4486-a899-e3019be9cc16 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35012025-03-04T03:20:14.938ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1915 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a0358e2-b53f-4486-a899-e3019be9cc16 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35022025-03-04T03:20:15.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1a61e7cb-59a5-441c-9dff-e9eea20823a2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35032025-03-04T03:20:15.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1a61e7cb-59a5-441c-9dff-e9eea20823a2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35042025-03-04T03:20:15.959ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2147 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1a61e7cb-59a5-441c-9dff-e9eea20823a2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35052025-03-04T03:20:16.977ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e3de612-073e-45da-afe2-4c110e224ad7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35062025-03-04T03:20:16.977ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e3de612-073e-45da-afe2-4c110e224ad7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35072025-03-04T03:20:16.979ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2175 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e3de612-073e-45da-afe2-4c110e224ad7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35082025-03-04T03:20:17.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
35092025-03-04T03:20:17.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35102025-03-04T03:20:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35112025-03-04T03:20:17.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
35122025-03-04T03:20:17.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35132025-03-04T03:20:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35142025-03-04T03:20:17.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35152025-03-04T03:20:17.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35162025-03-04T03:20:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35172025-03-04T03:20:17.207ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.825047ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35182025-03-04T03:20:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35192025-03-04T03:20:17.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35202025-03-04T03:20:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35212025-03-04T03:20:17.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35222025-03-04T03:20:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35232025-03-04T03:20:17.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35242025-03-04T03:20:17.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
35252025-03-04T03:20:17.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35262025-03-04T03:20:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35272025-03-04T03:20:17.224ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
35282025-03-04T03:20:17.224ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.438303ms iteration = 14 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
35292025-03-04T03:20:17.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 498 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = ee095f2b-9e3f-4d42-b211-be2a74876b4e response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
35302025-03-04T03:20:17.998ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7729fdb8-7e74-4e96-8e41-db3a6e91aa98 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35312025-03-04T03:20:17.998ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7729fdb8-7e74-4e96-8e41-db3a6e91aa98 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35322025-03-04T03:20:18.000ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2067 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7729fdb8-7e74-4e96-8e41-db3a6e91aa98 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35332025-03-04T03:20:19.019ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d76dbee-525e-4d5c-910f-fe51c3c8b9e3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35342025-03-04T03:20:19.019ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d76dbee-525e-4d5c-910f-fe51c3c8b9e3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35352025-03-04T03:20:19.021ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2146 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d76dbee-525e-4d5c-910f-fe51c3c8b9e3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35362025-03-04T03:20:20.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ac2fb8cf-367a-4374-962d-e51969ad07a0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35372025-03-04T03:20:20.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ac2fb8cf-367a-4374-962d-e51969ad07a0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35382025-03-04T03:20:20.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2262 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ac2fb8cf-367a-4374-962d-e51969ad07a0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35392025-03-04T03:20:21.062ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f9db381-cb79-47ba-89cc-4efe9d10908e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35402025-03-04T03:20:21.062ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f9db381-cb79-47ba-89cc-4efe9d10908e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35412025-03-04T03:20:21.064ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2495 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f9db381-cb79-47ba-89cc-4efe9d10908e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35422025-03-04T03:20:22.084ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 01c43522-7c63-4ab2-b9fa-67fe8316eb6b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35432025-03-04T03:20:22.084ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 01c43522-7c63-4ab2-b9fa-67fe8316eb6b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35442025-03-04T03:20:22.086ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2214 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 01c43522-7c63-4ab2-b9fa-67fe8316eb6b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35452025-03-04T03:20:23.106ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6d8425ea-d71e-457d-ba15-045514fd98fc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35462025-03-04T03:20:23.106ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6d8425ea-d71e-457d-ba15-045514fd98fc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35472025-03-04T03:20:23.108ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2207 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6d8425ea-d71e-457d-ba15-045514fd98fc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35482025-03-04T03:20:24.128ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 64fb22d3-b80e-4412-8578-21730254ac6e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35492025-03-04T03:20:24.129ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 64fb22d3-b80e-4412-8578-21730254ac6e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35502025-03-04T03:20:24.131ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2472 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 64fb22d3-b80e-4412-8578-21730254ac6e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35512025-03-04T03:20:25.149ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a3611ad-1b34-47f2-b3db-bf3e6c2e66e1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35522025-03-04T03:20:25.149ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a3611ad-1b34-47f2-b3db-bf3e6c2e66e1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35532025-03-04T03:20:25.152ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2836 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a3611ad-1b34-47f2-b3db-bf3e6c2e66e1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35542025-03-04T03:20:26.170ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ce87f859-cd09-4ad7-bf1e-329da5e75c14 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35552025-03-04T03:20:26.170ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ce87f859-cd09-4ad7-bf1e-329da5e75c14 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35562025-03-04T03:20:26.172ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1663 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ce87f859-cd09-4ad7-bf1e-329da5e75c14 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35572025-03-04T03:20:27.191ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 11b71e18-0fd5-48f5-94cf-2a8e2542a83f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35582025-03-04T03:20:27.191ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 11b71e18-0fd5-48f5-94cf-2a8e2542a83f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35592025-03-04T03:20:27.193ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2226 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 11b71e18-0fd5-48f5-94cf-2a8e2542a83f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35602025-03-04T03:20:27.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
35612025-03-04T03:20:27.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35622025-03-04T03:20:27.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35632025-03-04T03:20:27.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
35642025-03-04T03:20:27.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35652025-03-04T03:20:27.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35662025-03-04T03:20:27.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35672025-03-04T03:20:27.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35682025-03-04T03:20:27.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35692025-03-04T03:20:27.206ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.175029ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35702025-03-04T03:20:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35712025-03-04T03:20:27.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35722025-03-04T03:20:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35732025-03-04T03:20:27.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35742025-03-04T03:20:27.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35752025-03-04T03:20:27.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35762025-03-04T03:20:27.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
35772025-03-04T03:20:27.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35782025-03-04T03:20:27.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35792025-03-04T03:20:27.225ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
35802025-03-04T03:20:27.225ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.665638ms iteration = 15 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
35812025-03-04T03:20:27.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 760 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 6aa22a85-11b5-4857-9cea-aae4d0adafdc response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
35822025-03-04T03:20:28.212ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8508636b-836e-419c-86ba-215ea1da8a12 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35832025-03-04T03:20:28.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8508636b-836e-419c-86ba-215ea1da8a12 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35842025-03-04T03:20:28.214ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1627 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8508636b-836e-419c-86ba-215ea1da8a12 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35852025-03-04T03:20:29.233ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0ffe1ea-66d3-408d-be15-aea777ea2cbb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35862025-03-04T03:20:29.233ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0ffe1ea-66d3-408d-be15-aea777ea2cbb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35872025-03-04T03:20:29.235ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2228 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0ffe1ea-66d3-408d-be15-aea777ea2cbb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35882025-03-04T03:20:30.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 871fd4d5-aaf1-4885-be23-5151bb160211 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35892025-03-04T03:20:30.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 871fd4d5-aaf1-4885-be23-5151bb160211 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35902025-03-04T03:20:30.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2705 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 871fd4d5-aaf1-4885-be23-5151bb160211 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35912025-03-04T03:20:31.277ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed4b6eb7-fd28-4e0a-bdd8-7ae098c46e69 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35922025-03-04T03:20:31.277ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed4b6eb7-fd28-4e0a-bdd8-7ae098c46e69 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35932025-03-04T03:20:31.279ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2173 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed4b6eb7-fd28-4e0a-bdd8-7ae098c46e69 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35942025-03-04T03:20:32.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f85fa04-4e2b-49d8-bdc8-b4db6c97eaa0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35952025-03-04T03:20:32.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f85fa04-4e2b-49d8-bdc8-b4db6c97eaa0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35962025-03-04T03:20:32.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1973 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f85fa04-4e2b-49d8-bdc8-b4db6c97eaa0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35972025-03-04T03:20:33.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c689fc86-e370-458a-a915-4ca794967d7b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35982025-03-04T03:20:33.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c689fc86-e370-458a-a915-4ca794967d7b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
35992025-03-04T03:20:33.319ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1888 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c689fc86-e370-458a-a915-4ca794967d7b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36002025-03-04T03:20:34.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84e2574e-13e8-45ba-b198-857cddd4b7c4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36012025-03-04T03:20:34.341ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84e2574e-13e8-45ba-b198-857cddd4b7c4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36022025-03-04T03:20:34.343ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2464 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84e2574e-13e8-45ba-b198-857cddd4b7c4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36032025-03-04T03:20:35.362ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab543b0f-936e-4858-9c84-cae55498df04 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36042025-03-04T03:20:35.363ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab543b0f-936e-4858-9c84-cae55498df04 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36052025-03-04T03:20:35.364ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2495 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab543b0f-936e-4858-9c84-cae55498df04 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36062025-03-04T03:20:36.383ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f70a28bb-5639-4f51-9d4e-dbf346c716b5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36072025-03-04T03:20:36.384ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f70a28bb-5639-4f51-9d4e-dbf346c716b5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36082025-03-04T03:20:36.385ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1982 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f70a28bb-5639-4f51-9d4e-dbf346c716b5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
36092025-03-04T03:20:37.201ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
36102025-03-04T03:20:37.202ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36112025-03-04T03:20:37.202ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
36122025-03-04T03:20:37.202ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
36132025-03-04T03:20:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36142025-03-04T03:20:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
36152025-03-04T03:20:37.202ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
36162025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
36172025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
36182025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36192025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36202025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36212025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
36222025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 6 reason = Timeout
36232025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36242025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
36252025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
36262025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
36272025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
36282025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
36292025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36302025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36312025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36322025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36332025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
36342025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36352025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36362025-03-04T03:20:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36372025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
36382025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
36392025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
36402025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36412025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36422025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
36432025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
36442025-03-04T03:20:37.204ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
36452025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36462025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
36472025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36482025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36492025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36502025-03-04T03:20:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36512025-03-04T03:20:37.204ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1149, tv_nsec: 402416969 } })
36522025-03-04T03:20:37.204ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
36532025-03-04T03:20:37.205ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1149, tv_nsec: 403619124 } })
36542025-03-04T03:20:37.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36552025-03-04T03:20:37.205ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
36562025-03-04T03:20:37.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36572025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36582025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 6 reason = Timeout
36592025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
36602025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.485364ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
36612025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
36622025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36632025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
36642025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
36652025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
36662025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
36672025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36682025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
36692025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36702025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
36712025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36722025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
36732025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36742025-03-04T03:20:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
36752025-03-04T03:20:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
36762025-03-04T03:20:37.208ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36772025-03-04T03:20:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36782025-03-04T03:20:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
36792025-03-04T03:20:37.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36802025-03-04T03:20:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36812025-03-04T03:20:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
36822025-03-04T03:20:37.210ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
36832025-03-04T03:20:37.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36842025-03-04T03:20:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
36852025-03-04T03:20:37.210ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36862025-03-04T03:20:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
36872025-03-04T03:20:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36882025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36892025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36902025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36912025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36922025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36932025-03-04T03:20:37.212ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36942025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
36952025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36962025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36972025-03-04T03:20:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36982025-03-04T03:20:37.213ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36992025-03-04T03:20:37.213ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37002025-03-04T03:20:37.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37012025-03-04T03:20:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37022025-03-04T03:20:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
37032025-03-04T03:20:37.215ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37042025-03-04T03:20:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
37052025-03-04T03:20:37.215ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37062025-03-04T03:20:37.215ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 7.982558ms iteration = 6 status = {"errors":[],"finish_invoked_ok":[]}
37072025-03-04T03:20:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37082025-03-04T03:20:37.217ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 9.456536ms iteration = 6 status = {"errors":[],"requests_created_ok":[]}
37092025-03-04T03:20:37.217ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 10.299916ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
37102025-03-04T03:20:37.218ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37112025-03-04T03:20:37.218ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37122025-03-04T03:20:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
37132025-03-04T03:20:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37142025-03-04T03:20:37.218ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37152025-03-04T03:20:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37162025-03-04T03:20:37.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37172025-03-04T03:20:37.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37182025-03-04T03:20:37.219ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 15.901175ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37192025-03-04T03:20:37.220ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1149, tv_nsec: 418057011 } })
37202025-03-04T03:20:37.220ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
37212025-03-04T03:20:37.220ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37222025-03-04T03:20:37.221ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
37232025-03-04T03:20:37.221ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
37242025-03-04T03:20:37.221ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37252025-03-04T03:20:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
37262025-03-04T03:20:37.222ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 14.713381ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
37272025-03-04T03:20:37.222ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37282025-03-04T03:20:37.222ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37292025-03-04T03:20:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37302025-03-04T03:20:37.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37312025-03-04T03:20:37.225ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
37322025-03-04T03:20:37.225ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
37332025-03-04T03:20:37.225ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 21.265641ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
37342025-03-04T03:20:37.226ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
37352025-03-04T03:20:37.226ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 22.377194ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
37362025-03-04T03:20:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
37372025-03-04T03:20:37.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37382025-03-04T03:20:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37392025-03-04T03:20:37.228ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37402025-03-04T03:20:37.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37412025-03-04T03:20:37.230ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37422025-03-04T03:20:37.231ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37432025-03-04T03:20:37.231ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37442025-03-04T03:20:37.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37452025-03-04T03:20:37.233ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37462025-03-04T03:20:37.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37472025-03-04T03:20:37.235ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37482025-03-04T03:20:37.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37492025-03-04T03:20:37.235ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
37502025-03-04T03:20:37.235ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.476588ms iteration = 16 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
37512025-03-04T03:20:37.241ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37522025-03-04T03:20:37.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37532025-03-04T03:20:37.243ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37542025-03-04T03:20:37.243ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37552025-03-04T03:20:37.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37562025-03-04T03:20:37.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37572025-03-04T03:20:37.245ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37582025-03-04T03:20:37.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
37592025-03-04T03:20:37.245ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37602025-03-04T03:20:37.246ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37612025-03-04T03:20:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
37622025-03-04T03:20:37.248ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
37632025-03-04T03:20:37.248ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37642025-03-04T03:20:37.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37652025-03-04T03:20:37.250ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37662025-03-04T03:20:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37672025-03-04T03:20:37.251ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37682025-03-04T03:20:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37692025-03-04T03:20:37.292ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
37702025-03-04T03:20:37.292ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37712025-03-04T03:20:37.292ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d864f9a5-3042-461e-8691-f5999cbadc89", "content-length": "26", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
37722025-03-04T03:20:37.292ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37732025-03-04T03:20:37.293ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37742025-03-04T03:20:37.295ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
37752025-03-04T03:20:37.295ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37762025-03-04T03:20:37.296ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "69c0c2ba-733b-49c3-99f1-b2f65ecbaca1", "content-length": "26", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
37772025-03-04T03:20:37.296ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37782025-03-04T03:20:37.297ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37792025-03-04T03:20:37.327ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
37802025-03-04T03:20:37.327ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37812025-03-04T03:20:37.328ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bef4c9a2-d1f8-4b24-92cd-2c7597f7a65c", "content-length": "26", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
37822025-03-04T03:20:37.328ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37832025-03-04T03:20:37.329ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
37842025-03-04T03:20:37.404ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d2d058b3-3a15-4088-bb40-9501bbc8d63b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
37852025-03-04T03:20:37.404ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d2d058b3-3a15-4088-bb40-9501bbc8d63b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
37862025-03-04T03:20:37.425ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
37872025-03-04T03:20:37.425ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
37882025-03-04T03:20:37.425ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe742654-dfea-43f4-a1ce-d9badf771fd1", "content-length": "372", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
37892025-03-04T03:20:37.425ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37902025-03-04T03:20:37.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37912025-03-04T03:20:37.453ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
37922025-03-04T03:20:37.453ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "18bf35cb-15f4-4b4d-9fc9-db52f9be1f66", "content-length": "691", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
37932025-03-04T03:20:37.453ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
37942025-03-04T03:20:37.453ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
37952025-03-04T03:20:37.453ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
37962025-03-04T03:20:37.454ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 251.29217ms iteration = 6 status = {}
37972025-03-04T03:20:37.455ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37982025-03-04T03:20:37.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37992025-03-04T03:20:37.455ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 252.520535ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
38002025-03-04T03:20:37.455ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38012025-03-04T03:20:37.456ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 253.656608ms iteration = 6 status = 0
38022025-03-04T03:20:37.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 53077 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d2d058b3-3a15-4088-bb40-9501bbc8d63b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38032025-03-04T03:20:37.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
38042025-03-04T03:20:37.457ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38052025-03-04T03:20:37.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38062025-03-04T03:20:37.488ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 280.506537ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
38072025-03-04T03:20:37.492ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
38082025-03-04T03:20:37.492ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 289.552599ms iteration = 6 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
38092025-03-04T03:20:37.494ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 291.867467ms iteration = 8 status = {"physical_disks_added":0}
38102025-03-04T03:20:37.506ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
38112025-03-04T03:20:37.506ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
38122025-03-04T03:20:37.507ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e2928766-0681-4848-bcf1-12d99aca4673", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
38132025-03-04T03:20:37.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38142025-03-04T03:20:37.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38152025-03-04T03:20:37.509ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38162025-03-04T03:20:37.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38172025-03-04T03:20:37.512ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38182025-03-04T03:20:37.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38192025-03-04T03:20:37.515ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
38202025-03-04T03:20:37.515ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38212025-03-04T03:20:37.515ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "01c830d1-5809-43a0-96de-14c6d195fc6d", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
38222025-03-04T03:20:37.516ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 312.620282ms iteration = 6 status = {}
38232025-03-04T03:20:37.562ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 360.916643ms iteration = 6 status = {}
38242025-03-04T03:20:37.568ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
38252025-03-04T03:20:37.568ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "34f48d85-d6c2-4ee9-81a1-54f719775379", "content-length": "1", "date": "Tue, 04 Mar 2025 03:20:36 GMT"} })
38262025-03-04T03:20:37.569ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
38272025-03-04T03:20:37.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
38282025-03-04T03:20:37.571ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 370.033915ms iteration = 6 status = null
38292025-03-04T03:20:37.598ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38302025-03-04T03:20:37.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38312025-03-04T03:20:37.600ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
38322025-03-04T03:20:37.600ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38332025-03-04T03:20:37.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38342025-03-04T03:20:37.610ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
38352025-03-04T03:20:37.610ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "456af2df-11ae-41f8-8ee6-09a81e620b1d", "content-length": "134", "date": "Tue, 04 Mar 2025 03:20:36 GMT"} })
38362025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
38372025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
38382025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
38392025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
38402025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
38412025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
38422025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
38432025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
38442025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
38452025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
38462025-03-04T03:20:37.610ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
38472025-03-04T03:20:37.610ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38482025-03-04T03:20:37.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38492025-03-04T03:20:37.612ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
38502025-03-04T03:20:37.612ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "658a3f28-715c-4c8d-ab42-ef4698be56e8", "content-length": "2", "date": "Tue, 04 Mar 2025 03:20:36 GMT"} })
38512025-03-04T03:20:37.612ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
38522025-03-04T03:20:37.612ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2c43cbc-ad98-458e-8fa3-43ae2415cac2", "content-length": "51", "date": "Tue, 04 Mar 2025 03:20:36 GMT"} })
38532025-03-04T03:20:37.612ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
38542025-03-04T03:20:37.612ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
38552025-03-04T03:20:37.701ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6ddfe000-cb94-4ec0-bd65-5b8cbe9dec43", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
38562025-03-04T03:20:37.701ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
38572025-03-04T03:20:37.701ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
38582025-03-04T03:20:37.701ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "38a97617-a4f0-4e53-965e-29be3303785b", "content-length": "682", "date": "Tue, 04 Mar 2025 03:20:37 GMT"} })
38592025-03-04T03:20:37.701ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38602025-03-04T03:20:37.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38612025-03-04T03:20:37.705ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38622025-03-04T03:20:37.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38632025-03-04T03:20:37.708ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38642025-03-04T03:20:37.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38652025-03-04T03:20:37.711ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38662025-03-04T03:20:37.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38672025-03-04T03:20:37.718ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 515.799272ms iteration = 6 status = {}
38682025-03-04T03:20:37.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 442 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 501cbeb0-0ced-498d-b3a0-9d993458f479 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
38692025-03-04T03:20:38.476ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 59aad93c-9bd4-43bb-ad77-918c358047dc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38702025-03-04T03:20:38.479ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 59aad93c-9bd4-43bb-ad77-918c358047dc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38712025-03-04T03:20:38.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4774 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 59aad93c-9bd4-43bb-ad77-918c358047dc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38722025-03-04T03:20:39.506ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f61a7b7-29fe-4d54-b3af-58c4ea42bda9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38732025-03-04T03:20:39.506ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f61a7b7-29fe-4d54-b3af-58c4ea42bda9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38742025-03-04T03:20:39.509ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2841 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f61a7b7-29fe-4d54-b3af-58c4ea42bda9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38752025-03-04T03:20:40.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 977bcaf9-22e0-4780-87cd-a5320c7324de roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38762025-03-04T03:20:40.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 977bcaf9-22e0-4780-87cd-a5320c7324de resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38772025-03-04T03:20:40.528ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2184 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 977bcaf9-22e0-4780-87cd-a5320c7324de response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38782025-03-04T03:20:41.547ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1fe9f67-7294-4a21-8953-f5b661081173 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38792025-03-04T03:20:41.548ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1fe9f67-7294-4a21-8953-f5b661081173 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38802025-03-04T03:20:41.550ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2595 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1fe9f67-7294-4a21-8953-f5b661081173 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38812025-03-04T03:20:42.569ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bae89a50-934b-450c-a3d0-0ceb71b25fb0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38822025-03-04T03:20:42.569ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bae89a50-934b-450c-a3d0-0ceb71b25fb0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38832025-03-04T03:20:42.571ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2495 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bae89a50-934b-450c-a3d0-0ceb71b25fb0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38842025-03-04T03:20:43.589ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9e76f4b4-181d-44dd-8d4b-2727ba2ce0b4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38852025-03-04T03:20:43.590ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9e76f4b4-181d-44dd-8d4b-2727ba2ce0b4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38862025-03-04T03:20:43.592ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2575 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9e76f4b4-181d-44dd-8d4b-2727ba2ce0b4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38872025-03-04T03:20:44.612ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8b463afd-7a25-4de7-b202-c1ef2e063cb5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38882025-03-04T03:20:44.612ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8b463afd-7a25-4de7-b202-c1ef2e063cb5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38892025-03-04T03:20:44.614ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2608 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8b463afd-7a25-4de7-b202-c1ef2e063cb5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38902025-03-04T03:20:45.632ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eb0c4d54-8a59-4461-8c1c-13827d276c4f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38912025-03-04T03:20:45.633ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eb0c4d54-8a59-4461-8c1c-13827d276c4f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38922025-03-04T03:20:45.634ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1922 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eb0c4d54-8a59-4461-8c1c-13827d276c4f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38932025-03-04T03:20:46.653ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b8912ad-fa6f-49c5-91b7-64e5b3e1e0e2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38942025-03-04T03:20:46.653ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b8912ad-fa6f-49c5-91b7-64e5b3e1e0e2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38952025-03-04T03:20:46.655ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2155 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b8912ad-fa6f-49c5-91b7-64e5b3e1e0e2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
38962025-03-04T03:20:47.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
38972025-03-04T03:20:47.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38982025-03-04T03:20:47.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38992025-03-04T03:20:47.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
39002025-03-04T03:20:47.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39012025-03-04T03:20:47.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39022025-03-04T03:20:47.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39032025-03-04T03:20:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39042025-03-04T03:20:47.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39052025-03-04T03:20:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39062025-03-04T03:20:47.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39072025-03-04T03:20:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39082025-03-04T03:20:47.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39092025-03-04T03:20:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
39102025-03-04T03:20:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39112025-03-04T03:20:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39122025-03-04T03:20:47.224ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
39132025-03-04T03:20:47.224ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.947849ms iteration = 17 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
39142025-03-04T03:20:47.242ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39152025-03-04T03:20:47.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39162025-03-04T03:20:47.244ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.77683ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39172025-03-04T03:20:47.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22266bdb-d9d0-4d0b-b6ca-818bd883cab9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39182025-03-04T03:20:47.674ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22266bdb-d9d0-4d0b-b6ca-818bd883cab9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39192025-03-04T03:20:47.676ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2262 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22266bdb-d9d0-4d0b-b6ca-818bd883cab9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39202025-03-04T03:20:47.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 491 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 0f11105c-9e4d-43ea-89b0-eb6117fba1b5 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
39212025-03-04T03:20:48.694ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e8424ca-8056-4403-8604-6605858fe871 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39222025-03-04T03:20:48.695ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e8424ca-8056-4403-8604-6605858fe871 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39232025-03-04T03:20:48.696ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1954 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e8424ca-8056-4403-8604-6605858fe871 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39242025-03-04T03:20:49.715ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 30f01527-529e-47f3-a13f-ae6fe86d6b15 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39252025-03-04T03:20:49.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 30f01527-529e-47f3-a13f-ae6fe86d6b15 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39262025-03-04T03:20:49.717ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2095 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 30f01527-529e-47f3-a13f-ae6fe86d6b15 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39272025-03-04T03:20:50.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc55cfdb-0869-4b2c-b5f4-9b1760edcf6a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39282025-03-04T03:20:50.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc55cfdb-0869-4b2c-b5f4-9b1760edcf6a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39292025-03-04T03:20:50.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2269 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc55cfdb-0869-4b2c-b5f4-9b1760edcf6a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39302025-03-04T03:20:51.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb877341-d6e2-4abd-aff1-5318241a8c28 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39312025-03-04T03:20:51.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb877341-d6e2-4abd-aff1-5318241a8c28 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39322025-03-04T03:20:51.759ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1811 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb877341-d6e2-4abd-aff1-5318241a8c28 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39332025-03-04T03:20:52.780ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 383a0c1f-47f1-4bbe-91b6-8b82fdad1923 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39342025-03-04T03:20:52.780ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 383a0c1f-47f1-4bbe-91b6-8b82fdad1923 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39352025-03-04T03:20:52.782ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2440 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 383a0c1f-47f1-4bbe-91b6-8b82fdad1923 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39362025-03-04T03:20:53.801ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e877594d-33d2-45f5-937e-d2d5ae869bd2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39372025-03-04T03:20:53.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e877594d-33d2-45f5-937e-d2d5ae869bd2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39382025-03-04T03:20:53.803ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2225 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e877594d-33d2-45f5-937e-d2d5ae869bd2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39392025-03-04T03:20:54.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c18db1c9-15ba-4289-9cfd-9f8ce33d9058 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39402025-03-04T03:20:54.823ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c18db1c9-15ba-4289-9cfd-9f8ce33d9058 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39412025-03-04T03:20:54.825ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2157 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c18db1c9-15ba-4289-9cfd-9f8ce33d9058 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39422025-03-04T03:20:55.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7e6ecbf-b4db-43b1-8bd0-181f410f0de0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39432025-03-04T03:20:55.844ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7e6ecbf-b4db-43b1-8bd0-181f410f0de0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39442025-03-04T03:20:55.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2327 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7e6ecbf-b4db-43b1-8bd0-181f410f0de0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39452025-03-04T03:20:56.864ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc260519-eb05-49c2-a8d3-d02f7ffeef90 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39462025-03-04T03:20:56.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc260519-eb05-49c2-a8d3-d02f7ffeef90 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39472025-03-04T03:20:56.866ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1936 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc260519-eb05-49c2-a8d3-d02f7ffeef90 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39482025-03-04T03:20:57.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
39492025-03-04T03:20:57.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39502025-03-04T03:20:57.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39512025-03-04T03:20:57.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
39522025-03-04T03:20:57.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39532025-03-04T03:20:57.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39542025-03-04T03:20:57.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39552025-03-04T03:20:57.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39562025-03-04T03:20:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39572025-03-04T03:20:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39582025-03-04T03:20:57.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39592025-03-04T03:20:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39602025-03-04T03:20:57.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39612025-03-04T03:20:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39622025-03-04T03:20:57.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39632025-03-04T03:20:57.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
39642025-03-04T03:20:57.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39652025-03-04T03:20:57.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39662025-03-04T03:20:57.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
39672025-03-04T03:20:57.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.124503ms iteration = 18 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
39682025-03-04T03:20:57.235ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.075391ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39692025-03-04T03:20:57.885ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a2755b66-1679-4fbe-841a-9685b72443e7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39702025-03-04T03:20:57.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a2755b66-1679-4fbe-841a-9685b72443e7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39712025-03-04T03:20:57.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2247 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a2755b66-1679-4fbe-841a-9685b72443e7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39722025-03-04T03:20:57.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 547 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 6f70f330-5282-4031-8016-2dc7b461f3d2 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
39732025-03-04T03:20:58.907ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e6c856d-7ef2-46b9-b78c-35c027fb845b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39742025-03-04T03:20:58.907ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e6c856d-7ef2-46b9-b78c-35c027fb845b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39752025-03-04T03:20:58.909ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2090 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e6c856d-7ef2-46b9-b78c-35c027fb845b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39762025-03-04T03:20:59.927ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 50f3b05a-ffbc-4f96-9b16-b414d92cfdae roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39772025-03-04T03:20:59.928ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 50f3b05a-ffbc-4f96-9b16-b414d92cfdae resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39782025-03-04T03:20:59.929ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1799 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 50f3b05a-ffbc-4f96-9b16-b414d92cfdae response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39792025-03-04T03:21:00.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b295fcaa-8ec2-401d-97d1-0880687d7822 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39802025-03-04T03:21:00.951ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b295fcaa-8ec2-401d-97d1-0880687d7822 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39812025-03-04T03:21:00.953ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2307 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b295fcaa-8ec2-401d-97d1-0880687d7822 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39822025-03-04T03:21:01.972ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4a07e53b-8e54-4fbe-bf86-fd0ca5f2c1ed roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39832025-03-04T03:21:01.972ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4a07e53b-8e54-4fbe-bf86-fd0ca5f2c1ed resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39842025-03-04T03:21:01.974ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4a07e53b-8e54-4fbe-bf86-fd0ca5f2c1ed response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39852025-03-04T03:21:02.888ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:33624
39862025-03-04T03:21:02.888ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = ddc78c8b-671f-4330-a717-9afe7d318a2d roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
39872025-03-04T03:21:02.889ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = ddc78c8b-671f-4330-a717-9afe7d318a2d resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
39882025-03-04T03:21:02.891ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2394 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = ddc78c8b-671f-4330-a717-9afe7d318a2d response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
39892025-03-04T03:21:02.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = baf62593-0b69-4078-bb88-aa57ee67a26e roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
39902025-03-04T03:21:02.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = baf62593-0b69-4078-bb88-aa57ee67a26e resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
39912025-03-04T03:21:02.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2498 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = baf62593-0b69-4078-bb88-aa57ee67a26e response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
39922025-03-04T03:21:02.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec89801e-957e-4244-9cd6-59c1567c8d30 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39932025-03-04T03:21:02.993ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec89801e-957e-4244-9cd6-59c1567c8d30 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39942025-03-04T03:21:02.995ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2219 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec89801e-957e-4244-9cd6-59c1567c8d30 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39952025-03-04T03:21:04.014ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f26f0c54-c673-4a5f-9197-f0124f3d50af roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39962025-03-04T03:21:04.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f26f0c54-c673-4a5f-9197-f0124f3d50af resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39972025-03-04T03:21:04.016ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1631 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f26f0c54-c673-4a5f-9197-f0124f3d50af response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39982025-03-04T03:21:05.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec1f4ecc-9b8d-4caf-8f9d-584e19fe4ce3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
39992025-03-04T03:21:05.036ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec1f4ecc-9b8d-4caf-8f9d-584e19fe4ce3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
40002025-03-04T03:21:05.038ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2039 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec1f4ecc-9b8d-4caf-8f9d-584e19fe4ce3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
40012025-03-04T03:21:06.057ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c6f17d49-6948-44d0-8110-50a0cfb91d26 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
40022025-03-04T03:21:06.058ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c6f17d49-6948-44d0-8110-50a0cfb91d26 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
40032025-03-04T03:21:06.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2195 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c6f17d49-6948-44d0-8110-50a0cfb91d26 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
40042025-03-04T03:21:07.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a5d74c1a-182d-49a7-9e03-6b61d82604b8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
40052025-03-04T03:21:07.080ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a5d74c1a-182d-49a7-9e03-6b61d82604b8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
40062025-03-04T03:21:07.082ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2319 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a5d74c1a-182d-49a7-9e03-6b61d82604b8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
40072025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
40082025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
40092025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
40102025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40112025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
40122025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
40132025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40142025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
40152025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
40162025-03-04T03:21:07.202ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:11:07.202641482 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
40172025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
40182025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
40192025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
40202025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
40212025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
40222025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
40232025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
40242025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40252025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
40262025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
40272025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
40282025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40292025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40302025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
40312025-03-04T03:21:07.202ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
40322025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
40332025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 17.26\u{b5}s iteration = 5 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
40342025-03-04T03:21:07.202ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
40352025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
40362025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
40372025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
40382025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
40392025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 7 reason = Timeout
40402025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40412025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40422025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40432025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
40442025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40452025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
40462025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
40472025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40482025-03-04T03:21:07.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40492025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40502025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40512025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
40522025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
40532025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
40542025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
40552025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40562025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40572025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
40582025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40592025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40602025-03-04T03:21:07.203ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1179, tv_nsec: 401201208 } })
40612025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40622025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40632025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40642025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40652025-03-04T03:21:07.203ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1179, tv_nsec: 401316319 } })
40662025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40672025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40682025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
40692025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
40702025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40712025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40722025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
40732025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
40742025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
40752025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40762025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
40772025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
40782025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
40792025-03-04T03:21:07.203ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
40802025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40812025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40822025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40832025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
40842025-03-04T03:21:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.383967ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
40852025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
40862025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.297716ms iteration = 4 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
40872025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40882025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40892025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
40902025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40912025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40922025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
40932025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40942025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40952025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40962025-03-04T03:21:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40972025-03-04T03:21:07.205ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40982025-03-04T03:21:07.206ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
40992025-03-04T03:21:07.206ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
41002025-03-04T03:21:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
41012025-03-04T03:21:07.206ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41022025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.382114ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
41032025-03-04T03:21:07.207ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:11:07.202641482 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
41042025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.592816ms iteration = 4 status = {"expiration":"2025-03-04T03:11:07.202641482Z","pruned":[]}
41052025-03-04T03:21:07.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1179, tv_nsec: 405022344 } })
41062025-03-04T03:21:07.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
41072025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 7 reason = Timeout
41082025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
41092025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
41102025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
41112025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
41122025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
41132025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
41142025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
41152025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
41162025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41172025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
41182025-03-04T03:21:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
41192025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
41202025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
41212025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
41222025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41232025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
41242025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
41252025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41262025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41272025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
41282025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
41292025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
41302025-03-04T03:21:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41312025-03-04T03:21:07.210ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1179, tv_nsec: 408294815 } })
41322025-03-04T03:21:07.210ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
41332025-03-04T03:21:07.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41342025-03-04T03:21:07.211ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41352025-03-04T03:21:07.212ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41362025-03-04T03:21:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41372025-03-04T03:21:07.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41382025-03-04T03:21:07.213ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41392025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41402025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41412025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41422025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
41432025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41442025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
41452025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41462025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41472025-03-04T03:21:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41482025-03-04T03:21:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41492025-03-04T03:21:07.215ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41502025-03-04T03:21:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41512025-03-04T03:21:07.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41522025-03-04T03:21:07.216ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41532025-03-04T03:21:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
41542025-03-04T03:21:07.217ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41552025-03-04T03:21:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41562025-03-04T03:21:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41572025-03-04T03:21:07.218ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41582025-03-04T03:21:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41592025-03-04T03:21:07.221ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41602025-03-04T03:21:07.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41612025-03-04T03:21:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
41622025-03-04T03:21:07.223ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41632025-03-04T03:21:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41642025-03-04T03:21:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41652025-03-04T03:21:07.224ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41662025-03-04T03:21:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41672025-03-04T03:21:07.229ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.739816ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
41682025-03-04T03:21:07.231ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
41692025-03-04T03:21:07.231ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41702025-03-04T03:21:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
41712025-03-04T03:21:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41722025-03-04T03:21:07.233ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41732025-03-04T03:21:07.236ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41742025-03-04T03:21:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41752025-03-04T03:21:07.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41762025-03-04T03:21:07.240ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
41772025-03-04T03:21:07.240ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41782025-03-04T03:21:07.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41792025-03-04T03:21:07.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41802025-03-04T03:21:07.244ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41812025-03-04T03:21:07.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41822025-03-04T03:21:07.245ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41832025-03-04T03:21:07.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
41842025-03-04T03:21:07.249ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41852025-03-04T03:21:07.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41862025-03-04T03:21:07.258ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1179, tv_nsec: 455983729 } })
41872025-03-04T03:21:07.258ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
41882025-03-04T03:21:07.265ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41892025-03-04T03:21:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41902025-03-04T03:21:07.267ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41912025-03-04T03:21:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41922025-03-04T03:21:07.270ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41932025-03-04T03:21:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41942025-03-04T03:21:07.270ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 62.900371ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
41952025-03-04T03:21:07.271ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41962025-03-04T03:21:07.273ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
41972025-03-04T03:21:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
41982025-03-04T03:21:07.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41992025-03-04T03:21:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
42002025-03-04T03:21:07.274ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42012025-03-04T03:21:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42022025-03-04T03:21:07.275ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42032025-03-04T03:21:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42042025-03-04T03:21:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
42052025-03-04T03:21:07.278ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
42062025-03-04T03:21:07.278ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
42072025-03-04T03:21:07.278ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 71.970202ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
42082025-03-04T03:21:07.279ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42092025-03-04T03:21:07.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42102025-03-04T03:21:07.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42112025-03-04T03:21:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42122025-03-04T03:21:07.281ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
42132025-03-04T03:21:07.281ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
42142025-03-04T03:21:07.281ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 78.3672ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
42152025-03-04T03:21:07.281ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
42162025-03-04T03:21:07.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42172025-03-04T03:21:07.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.960407ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
42182025-03-04T03:21:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42192025-03-04T03:21:07.281ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42202025-03-04T03:21:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42212025-03-04T03:21:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42222025-03-04T03:21:07.284ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42232025-03-04T03:21:07.284ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
42242025-03-04T03:21:07.284ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42252025-03-04T03:21:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42262025-03-04T03:21:07.284ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 82.133756ms iteration = 3 status = {"nsuccess":5}
42272025-03-04T03:21:07.285ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 77.49327ms iteration = 7 status = {"errors":[],"requests_created_ok":[]}
42282025-03-04T03:21:07.287ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42292025-03-04T03:21:07.287ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
42302025-03-04T03:21:07.287ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42312025-03-04T03:21:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42322025-03-04T03:21:07.288ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
42332025-03-04T03:21:07.288ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
42342025-03-04T03:21:07.288ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
42352025-03-04T03:21:07.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42362025-03-04T03:21:07.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42372025-03-04T03:21:07.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42382025-03-04T03:21:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42392025-03-04T03:21:07.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42402025-03-04T03:21:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42412025-03-04T03:21:07.295ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
42422025-03-04T03:21:07.295ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
42432025-03-04T03:21:07.295ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42442025-03-04T03:21:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42452025-03-04T03:21:07.297ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42462025-03-04T03:21:07.300ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42472025-03-04T03:21:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42482025-03-04T03:21:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42492025-03-04T03:21:07.302ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42502025-03-04T03:21:07.302ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 94.410877ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
42512025-03-04T03:21:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42522025-03-04T03:21:07.302ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42532025-03-04T03:21:07.304ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
42542025-03-04T03:21:07.304ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
42552025-03-04T03:21:07.304ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
42562025-03-04T03:21:07.304ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 102.183433ms iteration = 4 status = {"generation":1}
42572025-03-04T03:21:07.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42582025-03-04T03:21:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
42592025-03-04T03:21:07.305ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42602025-03-04T03:21:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42612025-03-04T03:21:07.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
42622025-03-04T03:21:07.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42632025-03-04T03:21:07.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42642025-03-04T03:21:07.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 110.244902ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
42652025-03-04T03:21:07.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
42662025-03-04T03:21:07.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42672025-03-04T03:21:07.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
42682025-03-04T03:21:07.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42692025-03-04T03:21:07.322ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42702025-03-04T03:21:07.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42712025-03-04T03:21:07.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42722025-03-04T03:21:07.325ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
42732025-03-04T03:21:07.325ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 121.323667ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
42742025-03-04T03:21:07.325ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42752025-03-04T03:21:07.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42762025-03-04T03:21:07.325ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
42772025-03-04T03:21:07.326ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
42782025-03-04T03:21:07.326ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 122.424311ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42792025-03-04T03:21:07.326ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42802025-03-04T03:21:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42812025-03-04T03:21:07.326ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
42822025-03-04T03:21:07.326ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
42832025-03-04T03:21:07.326ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
42842025-03-04T03:21:07.326ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42852025-03-04T03:21:07.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42862025-03-04T03:21:07.327ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42872025-03-04T03:21:07.328ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42882025-03-04T03:21:07.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
42892025-03-04T03:21:07.328ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42902025-03-04T03:21:07.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42912025-03-04T03:21:07.329ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
42922025-03-04T03:21:07.329ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42932025-03-04T03:21:07.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42942025-03-04T03:21:07.330ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
42952025-03-04T03:21:07.330ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 127.318251ms iteration = 19 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
42962025-03-04T03:21:07.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
42972025-03-04T03:21:07.330ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42982025-03-04T03:21:07.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42992025-03-04T03:21:07.331ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43002025-03-04T03:21:07.332ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
43012025-03-04T03:21:07.332ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
43022025-03-04T03:21:07.332ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 128.302142ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
43032025-03-04T03:21:07.332ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 129.583558ms iteration = 7 status = 0
43042025-03-04T03:21:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
43052025-03-04T03:21:07.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43062025-03-04T03:21:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43072025-03-04T03:21:07.350ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
43082025-03-04T03:21:07.350ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 147.926243ms iteration = 7 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
43092025-03-04T03:21:07.356ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
43102025-03-04T03:21:07.356ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43112025-03-04T03:21:07.356ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e806d052-f1fd-48fb-96b8-8504a75f402b", "content-length": "26", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
43122025-03-04T03:21:07.356ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43132025-03-04T03:21:07.358ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43142025-03-04T03:21:07.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 150.581535ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
43152025-03-04T03:21:07.360ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
43162025-03-04T03:21:07.360ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43172025-03-04T03:21:07.360ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bad0a04-9ee3-4163-921e-e0e453c68c61", "content-length": "26", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
43182025-03-04T03:21:07.360ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43192025-03-04T03:21:07.361ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43202025-03-04T03:21:07.362ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43212025-03-04T03:21:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43222025-03-04T03:21:07.369ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
43232025-03-04T03:21:07.369ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
43242025-03-04T03:21:07.369ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43252025-03-04T03:21:07.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43262025-03-04T03:21:07.371ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43272025-03-04T03:21:07.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
43282025-03-04T03:21:07.374ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43292025-03-04T03:21:07.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43302025-03-04T03:21:07.376ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
43312025-03-04T03:21:07.376ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 174.232686ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
43322025-03-04T03:21:07.378ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
43332025-03-04T03:21:07.378ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
43342025-03-04T03:21:07.378ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
43352025-03-04T03:21:07.378ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 176.2729ms iteration = 4 status = {"generation":2}
43362025-03-04T03:21:07.395ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 193.074146ms iteration = 9 status = {"physical_disks_added":0}
43372025-03-04T03:21:07.402ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
43382025-03-04T03:21:07.402ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43392025-03-04T03:21:07.402ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d219e0e-5192-4efe-99bb-5ede5e4eddb8", "content-length": "26", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
43402025-03-04T03:21:07.402ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43412025-03-04T03:21:07.404ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
43422025-03-04T03:21:07.433ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
43432025-03-04T03:21:07.433ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 229.502453ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
43442025-03-04T03:21:07.463ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43452025-03-04T03:21:07.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43462025-03-04T03:21:07.464ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43472025-03-04T03:21:07.466ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
43482025-03-04T03:21:07.466ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43492025-03-04T03:21:07.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43502025-03-04T03:21:07.470ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
43512025-03-04T03:21:07.471ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "eae6316c-f97d-4f51-b136-f697cb603beb", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43522025-03-04T03:21:07.471ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43532025-03-04T03:21:07.472ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f30bb774-27ab-4f75-89f1-977b72176207", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43542025-03-04T03:21:07.472ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43552025-03-04T03:21:07.473ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43562025-03-04T03:21:07.473ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 270.804068ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
43572025-03-04T03:21:07.493ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
43582025-03-04T03:21:07.509ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 306.682659ms iteration = 7 status = {}
43592025-03-04T03:21:07.592ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
43602025-03-04T03:21:07.592ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab628a9d-0636-45f5-acf3-6d5867037c05", "content-length": "1", "date": "Tue, 04 Mar 2025 03:21:06 GMT"} })
43612025-03-04T03:21:07.594ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
43622025-03-04T03:21:07.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
43632025-03-04T03:21:07.603ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
43642025-03-04T03:21:07.603ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "f468d3cf-be43-4b87-8bec-e87e3939a23b", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43652025-03-04T03:21:07.603ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43662025-03-04T03:21:07.605ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c98f9007-c9ff-4fb2-8d58-c4180790d106", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43672025-03-04T03:21:07.605ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43682025-03-04T03:21:07.606ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "0757b05f-7d73-47a1-b438-2465d63ba6b0", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43692025-03-04T03:21:07.606ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43702025-03-04T03:21:07.608ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43712025-03-04T03:21:07.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 405.658061ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
43722025-03-04T03:21:07.614ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
43732025-03-04T03:21:07.614ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
43742025-03-04T03:21:07.615ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a909af5-f33f-418a-b4c9-64b7e4c76a47", "content-length": "372", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
43752025-03-04T03:21:07.615ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43762025-03-04T03:21:07.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43772025-03-04T03:21:07.616ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43782025-03-04T03:21:07.617ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 414.447199ms iteration = 7 status = null
43792025-03-04T03:21:07.618ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
43802025-03-04T03:21:07.618ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43812025-03-04T03:21:07.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43822025-03-04T03:21:07.634ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
43832025-03-04T03:21:07.634ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
43842025-03-04T03:21:07.634ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3d8282ce-b6c3-4698-b579-9bc9bf300b7a", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
43852025-03-04T03:21:07.634ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43862025-03-04T03:21:07.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43872025-03-04T03:21:07.637ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43882025-03-04T03:21:07.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43892025-03-04T03:21:07.640ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43902025-03-04T03:21:07.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43912025-03-04T03:21:07.643ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
43922025-03-04T03:21:07.643ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
43932025-03-04T03:21:07.643ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c5683189-1d1d-46d5-bfe7-d86c0fcdf01c", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
43942025-03-04T03:21:07.645ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 441.517151ms iteration = 7 status = {}
43952025-03-04T03:21:07.681ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
43962025-03-04T03:21:07.681ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "76376396-ecc9-4474-9d56-26f2c8c289f6", "content-length": "691", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
43972025-03-04T03:21:07.681ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
43982025-03-04T03:21:07.681ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
43992025-03-04T03:21:07.681ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
44002025-03-04T03:21:07.682ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 479.306414ms iteration = 7 status = {}
44012025-03-04T03:21:07.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44022025-03-04T03:21:07.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44032025-03-04T03:21:07.684ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
44042025-03-04T03:21:07.684ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44052025-03-04T03:21:07.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44062025-03-04T03:21:07.694ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
44072025-03-04T03:21:07.694ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9ff6c6d-a135-4f60-8ed7-df41f920ff49", "content-length": "134", "date": "Tue, 04 Mar 2025 03:21:06 GMT"} })
44082025-03-04T03:21:07.694ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
44092025-03-04T03:21:07.694ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
44102025-03-04T03:21:07.694ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
44112025-03-04T03:21:07.694ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
44122025-03-04T03:21:07.694ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
44132025-03-04T03:21:07.694ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
44142025-03-04T03:21:07.694ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
44152025-03-04T03:21:07.694ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
44162025-03-04T03:21:07.695ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
44172025-03-04T03:21:07.695ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
44182025-03-04T03:21:07.695ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
44192025-03-04T03:21:07.695ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44202025-03-04T03:21:07.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44212025-03-04T03:21:07.697ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
44222025-03-04T03:21:07.697ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e3a2322-b6ea-40c2-a42e-8d2f111cf33d", "content-length": "2", "date": "Tue, 04 Mar 2025 03:21:06 GMT"} })
44232025-03-04T03:21:07.697ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
44242025-03-04T03:21:07.697ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1db8c04-8570-4b34-9e74-a10a6ad679e2", "content-length": "51", "date": "Tue, 04 Mar 2025 03:21:06 GMT"} })
44252025-03-04T03:21:07.697ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
44262025-03-04T03:21:07.697ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
44272025-03-04T03:21:07.784ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1af8e31c-bf26-434e-a6b9-f25a4d195f80", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
44282025-03-04T03:21:07.784ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
44292025-03-04T03:21:07.784ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
44302025-03-04T03:21:07.784ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "69b98726-3c35-459c-9a92-8c347e2f7c37", "content-length": "682", "date": "Tue, 04 Mar 2025 03:21:07 GMT"} })
44312025-03-04T03:21:07.784ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44322025-03-04T03:21:07.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44332025-03-04T03:21:07.787ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44342025-03-04T03:21:07.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44352025-03-04T03:21:07.790ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44362025-03-04T03:21:07.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44372025-03-04T03:21:07.792ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44382025-03-04T03:21:07.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44392025-03-04T03:21:07.799ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 596.609882ms iteration = 7 status = {}
44402025-03-04T03:21:07.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 527 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 2c921423-d126-4569-a7ea-c957dc4be925 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
44412025-03-04T03:21:08.100ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f171850-ddd6-4372-857a-7f32b7368841 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44422025-03-04T03:21:08.101ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f171850-ddd6-4372-857a-7f32b7368841 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44432025-03-04T03:21:08.103ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2349 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f171850-ddd6-4372-857a-7f32b7368841 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44442025-03-04T03:21:09.122ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e58c78ff-d4fa-456d-8e96-638678517f6b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44452025-03-04T03:21:09.123ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e58c78ff-d4fa-456d-8e96-638678517f6b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44462025-03-04T03:21:09.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2213 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e58c78ff-d4fa-456d-8e96-638678517f6b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44472025-03-04T03:21:10.143ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3098ae58-5084-4384-bfc2-f4a870fe1f8c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44482025-03-04T03:21:10.143ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3098ae58-5084-4384-bfc2-f4a870fe1f8c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44492025-03-04T03:21:10.145ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2193 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3098ae58-5084-4384-bfc2-f4a870fe1f8c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44502025-03-04T03:21:11.164ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4077839c-dbae-459d-879a-1caec1a8a8da roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44512025-03-04T03:21:11.164ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4077839c-dbae-459d-879a-1caec1a8a8da resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44522025-03-04T03:21:11.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2665 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4077839c-dbae-459d-879a-1caec1a8a8da response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44532025-03-04T03:21:12.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 353c5694-f684-4acc-9949-3992a3b46092 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44542025-03-04T03:21:12.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 353c5694-f684-4acc-9949-3992a3b46092 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44552025-03-04T03:21:12.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2879 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 353c5694-f684-4acc-9949-3992a3b46092 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44562025-03-04T03:21:13.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e318daf4-272b-4761-9ddd-923969f7933a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44572025-03-04T03:21:13.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e318daf4-272b-4761-9ddd-923969f7933a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44582025-03-04T03:21:13.210ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e318daf4-272b-4761-9ddd-923969f7933a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44592025-03-04T03:21:14.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ea349586-fc7f-40e7-8845-52e82807128a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44602025-03-04T03:21:14.230ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ea349586-fc7f-40e7-8845-52e82807128a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44612025-03-04T03:21:14.232ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2341 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ea349586-fc7f-40e7-8845-52e82807128a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44622025-03-04T03:21:15.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12bcb966-36b2-4f0c-8ecc-db16b0233e31 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44632025-03-04T03:21:15.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12bcb966-36b2-4f0c-8ecc-db16b0233e31 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44642025-03-04T03:21:15.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1746 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12bcb966-36b2-4f0c-8ecc-db16b0233e31 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44652025-03-04T03:21:16.272ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dd330747-0c25-4560-bc15-b1285969010d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44662025-03-04T03:21:16.273ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dd330747-0c25-4560-bc15-b1285969010d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44672025-03-04T03:21:16.275ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2066 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dd330747-0c25-4560-bc15-b1285969010d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44682025-03-04T03:21:17.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
44692025-03-04T03:21:17.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44702025-03-04T03:21:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44712025-03-04T03:21:17.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
44722025-03-04T03:21:17.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44732025-03-04T03:21:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44742025-03-04T03:21:17.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44752025-03-04T03:21:17.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44762025-03-04T03:21:17.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
44772025-03-04T03:21:17.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44782025-03-04T03:21:17.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44792025-03-04T03:21:17.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44802025-03-04T03:21:17.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44812025-03-04T03:21:17.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44822025-03-04T03:21:17.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44832025-03-04T03:21:17.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
44842025-03-04T03:21:17.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44852025-03-04T03:21:17.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44862025-03-04T03:21:17.257ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
44872025-03-04T03:21:17.257ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.376867ms iteration = 20 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
44882025-03-04T03:21:17.267ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.695291ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44892025-03-04T03:21:17.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 79ad7e9d-bb71-46fe-9e2b-84cec52d3a26 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44902025-03-04T03:21:17.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 79ad7e9d-bb71-46fe-9e2b-84cec52d3a26 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44912025-03-04T03:21:17.295ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2080 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 79ad7e9d-bb71-46fe-9e2b-84cec52d3a26 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
44922025-03-04T03:21:17.888ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = 1db40acb-5a82-40a0-9319-5c607603084b roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
44932025-03-04T03:21:17.888ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = 1db40acb-5a82-40a0-9319-5c607603084b resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
44942025-03-04T03:21:17.891ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3383 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = 1db40acb-5a82-40a0-9319-5c607603084b response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
44952025-03-04T03:21:17.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = aa4ea4a1-1c3e-46b8-9edd-2d83b2092b31 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
44962025-03-04T03:21:17.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = aa4ea4a1-1c3e-46b8-9edd-2d83b2092b31 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
44972025-03-04T03:21:17.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1987 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = aa4ea4a1-1c3e-46b8-9edd-2d83b2092b31 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
44982025-03-04T03:21:17.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 410 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = c3976d63-fb4e-4e1a-91e4-e2faed3b74dc response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
44992025-03-04T03:21:18.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4bc9b0b7-f720-4085-bfee-0387eed0fb73 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45002025-03-04T03:21:18.315ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4bc9b0b7-f720-4085-bfee-0387eed0fb73 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45012025-03-04T03:21:18.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2038 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4bc9b0b7-f720-4085-bfee-0387eed0fb73 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45022025-03-04T03:21:19.335ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68cec7b1-832d-4bcd-8d72-86c866e98a93 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45032025-03-04T03:21:19.336ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68cec7b1-832d-4bcd-8d72-86c866e98a93 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45042025-03-04T03:21:19.338ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2467 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68cec7b1-832d-4bcd-8d72-86c866e98a93 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45052025-03-04T03:21:20.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 50682476-45cb-4b92-810d-a1023523767e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45062025-03-04T03:21:20.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 50682476-45cb-4b92-810d-a1023523767e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45072025-03-04T03:21:20.359ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1938 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 50682476-45cb-4b92-810d-a1023523767e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45082025-03-04T03:21:21.378ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c19ead0d-a8d1-4b58-9c34-7f774f62f1bd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45092025-03-04T03:21:21.378ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c19ead0d-a8d1-4b58-9c34-7f774f62f1bd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45102025-03-04T03:21:21.379ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1685 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c19ead0d-a8d1-4b58-9c34-7f774f62f1bd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45112025-03-04T03:21:22.399ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6d83aee9-5d0f-45d4-9f28-b2803ae39665 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45122025-03-04T03:21:22.400ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6d83aee9-5d0f-45d4-9f28-b2803ae39665 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45132025-03-04T03:21:22.402ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2702 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6d83aee9-5d0f-45d4-9f28-b2803ae39665 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45142025-03-04T03:21:23.420ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93a78061-c683-4c03-b06d-fff7817a7635 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45152025-03-04T03:21:23.421ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93a78061-c683-4c03-b06d-fff7817a7635 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45162025-03-04T03:21:23.422ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2201 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93a78061-c683-4c03-b06d-fff7817a7635 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45172025-03-04T03:21:24.442ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7a5b082-f67b-46c6-973c-e47e3936e9b0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45182025-03-04T03:21:24.442ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7a5b082-f67b-46c6-973c-e47e3936e9b0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45192025-03-04T03:21:24.444ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2543 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7a5b082-f67b-46c6-973c-e47e3936e9b0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45202025-03-04T03:21:25.464ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a513ec4-ace6-4879-b94a-a98d48a89a40 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45212025-03-04T03:21:25.464ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a513ec4-ace6-4879-b94a-a98d48a89a40 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45222025-03-04T03:21:25.465ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1883 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a513ec4-ace6-4879-b94a-a98d48a89a40 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45232025-03-04T03:21:26.485ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9e35764-0682-419d-972d-96e3a6b8f262 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45242025-03-04T03:21:26.485ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9e35764-0682-419d-972d-96e3a6b8f262 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45252025-03-04T03:21:26.487ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2023 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9e35764-0682-419d-972d-96e3a6b8f262 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45262025-03-04T03:21:27.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
45272025-03-04T03:21:27.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45282025-03-04T03:21:27.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45292025-03-04T03:21:27.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
45302025-03-04T03:21:27.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45312025-03-04T03:21:27.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45322025-03-04T03:21:27.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45332025-03-04T03:21:27.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45342025-03-04T03:21:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45352025-03-04T03:21:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45362025-03-04T03:21:27.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45372025-03-04T03:21:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45382025-03-04T03:21:27.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45392025-03-04T03:21:27.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45402025-03-04T03:21:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45412025-03-04T03:21:27.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
45422025-03-04T03:21:27.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45432025-03-04T03:21:27.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45442025-03-04T03:21:27.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
45452025-03-04T03:21:27.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.608626ms iteration = 21 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
45462025-03-04T03:21:27.238ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.591293ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45472025-03-04T03:21:27.506ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a682e227-c243-4bd1-85df-08085cd37805 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45482025-03-04T03:21:27.506ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a682e227-c243-4bd1-85df-08085cd37805 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45492025-03-04T03:21:27.507ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1774 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a682e227-c243-4bd1-85df-08085cd37805 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45502025-03-04T03:21:27.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 617 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 2037210a-56ae-48a5-83e2-0ae6c07a62c5 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
45512025-03-04T03:21:28.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f91efefe-11a3-4416-9bf6-49fdbb66ec9d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45522025-03-04T03:21:28.527ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f91efefe-11a3-4416-9bf6-49fdbb66ec9d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45532025-03-04T03:21:28.528ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2062 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f91efefe-11a3-4416-9bf6-49fdbb66ec9d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45542025-03-04T03:21:29.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1de31b5d-bf90-4835-92b1-5acc1722b00e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45552025-03-04T03:21:29.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1de31b5d-bf90-4835-92b1-5acc1722b00e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45562025-03-04T03:21:29.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2209 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1de31b5d-bf90-4835-92b1-5acc1722b00e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45572025-03-04T03:21:30.570ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 74c0237c-13bb-4872-9c83-b81514ffbc43 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45582025-03-04T03:21:30.571ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 74c0237c-13bb-4872-9c83-b81514ffbc43 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45592025-03-04T03:21:30.572ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2031 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 74c0237c-13bb-4872-9c83-b81514ffbc43 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45602025-03-04T03:21:31.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4243a7ab-dc16-4715-880d-2ed42bb88d4b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45612025-03-04T03:21:31.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4243a7ab-dc16-4715-880d-2ed42bb88d4b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45622025-03-04T03:21:31.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2239 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4243a7ab-dc16-4715-880d-2ed42bb88d4b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45632025-03-04T03:21:32.614ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3e790acb-c440-4eff-8cf7-5da6e5a5f8e5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45642025-03-04T03:21:32.615ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3e790acb-c440-4eff-8cf7-5da6e5a5f8e5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45652025-03-04T03:21:32.616ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2056 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3e790acb-c440-4eff-8cf7-5da6e5a5f8e5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45662025-03-04T03:21:32.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = a53a223d-4542-4f30-876e-952f402bccda roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
45672025-03-04T03:21:32.889ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = a53a223d-4542-4f30-876e-952f402bccda resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
45682025-03-04T03:21:32.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3278 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = a53a223d-4542-4f30-876e-952f402bccda response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
45692025-03-04T03:21:32.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = c6c568b2-ad3f-4d88-8ad3-ecb5b9393f3b roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
45702025-03-04T03:21:32.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = c6c568b2-ad3f-4d88-8ad3-ecb5b9393f3b resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
45712025-03-04T03:21:32.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2434 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33624 req_id = c6c568b2-ad3f-4d88-8ad3-ecb5b9393f3b response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
45722025-03-04T03:21:33.635ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 241d2bd2-e2ef-4d8b-98ba-3d99eef83199 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45732025-03-04T03:21:33.636ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 241d2bd2-e2ef-4d8b-98ba-3d99eef83199 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45742025-03-04T03:21:33.637ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1771 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 241d2bd2-e2ef-4d8b-98ba-3d99eef83199 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45752025-03-04T03:21:34.657ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a0a2138-5693-4dac-879b-e71418b6b7a6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45762025-03-04T03:21:34.658ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a0a2138-5693-4dac-879b-e71418b6b7a6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45772025-03-04T03:21:34.659ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1791 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a0a2138-5693-4dac-879b-e71418b6b7a6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45782025-03-04T03:21:35.678ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a24895c8-c230-42c3-8a84-6aa2ccfc64dc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45792025-03-04T03:21:35.679ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a24895c8-c230-42c3-8a84-6aa2ccfc64dc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45802025-03-04T03:21:35.681ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2387 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a24895c8-c230-42c3-8a84-6aa2ccfc64dc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45812025-03-04T03:21:36.701ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a1d97436-22e4-4111-b08f-9fbd33188dbd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45822025-03-04T03:21:36.701ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a1d97436-22e4-4111-b08f-9fbd33188dbd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45832025-03-04T03:21:36.703ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2297 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a1d97436-22e4-4111-b08f-9fbd33188dbd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
45842025-03-04T03:21:37.202ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
45852025-03-04T03:21:37.202ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
45862025-03-04T03:21:37.202ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
45872025-03-04T03:21:37.202ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
45882025-03-04T03:21:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
45892025-03-04T03:21:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
45902025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
45912025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
45922025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45932025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
45942025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
45952025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45962025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
45972025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
45982025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45992025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
46002025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 8 reason = Timeout
46012025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46022025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
46032025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46042025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46052025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46062025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46072025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
46082025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46092025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46102025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
46112025-03-04T03:21:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46122025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
46132025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
46142025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
46152025-03-04T03:21:37.204ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
46162025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
46172025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
46182025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
46192025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46202025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
46212025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46222025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
46232025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46242025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46252025-03-04T03:21:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
46262025-03-04T03:21:37.204ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1209, tv_nsec: 402097900 } })
46272025-03-04T03:21:37.204ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
46282025-03-04T03:21:37.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46292025-03-04T03:21:37.205ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1209, tv_nsec: 402884370 } })
46302025-03-04T03:21:37.205ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
46312025-03-04T03:21:37.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46322025-03-04T03:21:37.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46332025-03-04T03:21:37.206ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.507213ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
46342025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
46352025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46362025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 8 reason = Timeout
46372025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
46382025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
46392025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
46402025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
46412025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
46422025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
46432025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
46442025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46452025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
46462025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
46472025-03-04T03:21:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46482025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
46492025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
46502025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
46512025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
46522025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
46532025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46542025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
46552025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46562025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46572025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46582025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46592025-03-04T03:21:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46602025-03-04T03:21:37.210ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46612025-03-04T03:21:37.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46622025-03-04T03:21:37.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46632025-03-04T03:21:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46642025-03-04T03:21:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46652025-03-04T03:21:37.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46662025-03-04T03:21:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46672025-03-04T03:21:37.213ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46682025-03-04T03:21:37.213ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46692025-03-04T03:21:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46702025-03-04T03:21:37.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46712025-03-04T03:21:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46722025-03-04T03:21:37.214ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.218976ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
46732025-03-04T03:21:37.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46742025-03-04T03:21:37.218ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1209, tv_nsec: 415775798 } })
46752025-03-04T03:21:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46762025-03-04T03:21:37.218ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
46772025-03-04T03:21:37.218ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46782025-03-04T03:21:37.219ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46792025-03-04T03:21:37.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46802025-03-04T03:21:37.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46812025-03-04T03:21:37.220ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46822025-03-04T03:21:37.220ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46832025-03-04T03:21:37.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46842025-03-04T03:21:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
46852025-03-04T03:21:37.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46862025-03-04T03:21:37.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46872025-03-04T03:21:37.224ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46882025-03-04T03:21:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46892025-03-04T03:21:37.226ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46902025-03-04T03:21:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46912025-03-04T03:21:37.250ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46922025-03-04T03:21:37.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46932025-03-04T03:21:37.252ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46942025-03-04T03:21:37.252ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46952025-03-04T03:21:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46962025-03-04T03:21:37.253ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
46972025-03-04T03:21:37.253ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.389077ms iteration = 22 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
46982025-03-04T03:21:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
46992025-03-04T03:21:37.254ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47002025-03-04T03:21:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47012025-03-04T03:21:37.255ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47022025-03-04T03:21:37.256ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47032025-03-04T03:21:37.256ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
47042025-03-04T03:21:37.256ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47052025-03-04T03:21:37.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47062025-03-04T03:21:37.256ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 53.470985ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
47072025-03-04T03:21:37.256ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47082025-03-04T03:21:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47092025-03-04T03:21:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
47102025-03-04T03:21:37.257ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
47112025-03-04T03:21:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
47122025-03-04T03:21:37.257ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 53.655117ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47132025-03-04T03:21:37.259ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 50.54295ms iteration = 8 status = {"errors":[],"requests_created_ok":[]}
47142025-03-04T03:21:37.259ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47152025-03-04T03:21:37.259ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
47162025-03-04T03:21:37.259ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47172025-03-04T03:21:37.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47182025-03-04T03:21:37.260ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 51.636973ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
47192025-03-04T03:21:37.261ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
47202025-03-04T03:21:37.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
47212025-03-04T03:21:37.263ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 55.001054ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
47222025-03-04T03:21:37.288ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47232025-03-04T03:21:37.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47242025-03-04T03:21:37.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47252025-03-04T03:21:37.294ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47262025-03-04T03:21:37.294ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
47272025-03-04T03:21:37.294ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 89.926842ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
47282025-03-04T03:21:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
47292025-03-04T03:21:37.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47302025-03-04T03:21:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47312025-03-04T03:21:37.294ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47322025-03-04T03:21:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47332025-03-04T03:21:37.295ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
47342025-03-04T03:21:37.295ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47352025-03-04T03:21:37.295ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "02763731-bfb0-4052-a428-eb906d2850a0", "content-length": "26", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
47362025-03-04T03:21:37.295ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47372025-03-04T03:21:37.296ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47382025-03-04T03:21:37.296ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 93.627297ms iteration = 8 status = 0
47392025-03-04T03:21:37.297ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47402025-03-04T03:21:37.297ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
47412025-03-04T03:21:37.297ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 93.725949ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
47422025-03-04T03:21:37.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47432025-03-04T03:21:37.299ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47442025-03-04T03:21:37.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47452025-03-04T03:21:37.301ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
47462025-03-04T03:21:37.301ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 97.958121ms iteration = 8 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
47472025-03-04T03:21:37.316ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
47482025-03-04T03:21:37.316ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47492025-03-04T03:21:37.316ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0bdac56-2808-4740-bdba-0b122047c6f5", "content-length": "26", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
47502025-03-04T03:21:37.316ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47512025-03-04T03:21:37.318ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
47522025-03-04T03:21:37.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 114.801137ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
47532025-03-04T03:21:37.334ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47542025-03-04T03:21:37.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47552025-03-04T03:21:37.338ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 135.586962ms iteration = 10 status = {"physical_disks_added":0}
47562025-03-04T03:21:37.339ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
47572025-03-04T03:21:37.339ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47582025-03-04T03:21:37.339ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f82f6b3-8e61-4677-8fde-c17bca422afc", "content-length": "26", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
47592025-03-04T03:21:37.339ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47602025-03-04T03:21:37.341ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47612025-03-04T03:21:37.342ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47622025-03-04T03:21:37.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47632025-03-04T03:21:37.407ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
47642025-03-04T03:21:37.407ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d8dd1ee-ebfc-4c1d-87ce-bd55d59a137e", "content-length": "1", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
47652025-03-04T03:21:37.409ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
47662025-03-04T03:21:37.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
47672025-03-04T03:21:37.423ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
47682025-03-04T03:21:37.423ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47692025-03-04T03:21:37.424ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2bd697a-2683-4a7a-94bb-26fd5b155695", "content-length": "372", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
47702025-03-04T03:21:37.424ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47712025-03-04T03:21:37.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47722025-03-04T03:21:37.425ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47732025-03-04T03:21:37.426ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 224.059316ms iteration = 8 status = null
47742025-03-04T03:21:37.427ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
47752025-03-04T03:21:37.427ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47762025-03-04T03:21:37.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47772025-03-04T03:21:37.428ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47782025-03-04T03:21:37.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
47792025-03-04T03:21:37.430ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47802025-03-04T03:21:37.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47812025-03-04T03:21:37.438ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 236.329276ms iteration = 8 status = {}
47822025-03-04T03:21:37.492ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
47832025-03-04T03:21:37.493ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "92d04149-e5e2-46a9-aa35-d5470129d162", "content-length": "691", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
47842025-03-04T03:21:37.493ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
47852025-03-04T03:21:37.493ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
47862025-03-04T03:21:37.493ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
47872025-03-04T03:21:37.494ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 291.069137ms iteration = 8 status = {}
47882025-03-04T03:21:37.515ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
47892025-03-04T03:21:37.515ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
47902025-03-04T03:21:37.515ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "411da034-bde2-4cdb-8b54-e88e4466bf62", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
47912025-03-04T03:21:37.515ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47922025-03-04T03:21:37.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47932025-03-04T03:21:37.517ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47942025-03-04T03:21:37.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47952025-03-04T03:21:37.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47962025-03-04T03:21:37.520ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47972025-03-04T03:21:37.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47982025-03-04T03:21:37.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47992025-03-04T03:21:37.522ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
48002025-03-04T03:21:37.522ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48012025-03-04T03:21:37.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48022025-03-04T03:21:37.524ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
48032025-03-04T03:21:37.524ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48042025-03-04T03:21:37.524ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7856161f-7654-41d2-965a-f351f4648aac", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
48052025-03-04T03:21:37.526ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 322.245599ms iteration = 8 status = {}
48062025-03-04T03:21:37.532ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
48072025-03-04T03:21:37.532ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ab78af2-f53e-4645-8673-a0a2f85cb36c", "content-length": "134", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
48082025-03-04T03:21:37.532ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
48092025-03-04T03:21:37.532ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
48102025-03-04T03:21:37.532ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
48112025-03-04T03:21:37.532ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
48122025-03-04T03:21:37.532ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48132025-03-04T03:21:37.532ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48142025-03-04T03:21:37.532ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48152025-03-04T03:21:37.532ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
48162025-03-04T03:21:37.533ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48172025-03-04T03:21:37.533ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
48182025-03-04T03:21:37.533ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
48192025-03-04T03:21:37.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48202025-03-04T03:21:37.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48212025-03-04T03:21:37.535ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
48222025-03-04T03:21:37.535ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "02b4e93c-74db-4d52-a230-f3faad817d94", "content-length": "2", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
48232025-03-04T03:21:37.535ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
48242025-03-04T03:21:37.535ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac42381f-2707-4f20-ae02-6d27e6053b80", "content-length": "51", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
48252025-03-04T03:21:37.535ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
48262025-03-04T03:21:37.535ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
48272025-03-04T03:21:37.628ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c046b285-bf40-46fd-9a23-66c46b9540f0", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
48282025-03-04T03:21:37.628ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
48292025-03-04T03:21:37.628ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
48302025-03-04T03:21:37.629ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e25c39c0-2e8b-436b-b379-0adb0f586837", "content-length": "682", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
48312025-03-04T03:21:37.629ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48322025-03-04T03:21:37.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48332025-03-04T03:21:37.632ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48342025-03-04T03:21:37.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48352025-03-04T03:21:37.635ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48362025-03-04T03:21:37.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48372025-03-04T03:21:37.637ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48382025-03-04T03:21:37.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48392025-03-04T03:21:37.644ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 441.202767ms iteration = 8 status = {}
48402025-03-04T03:21:37.722ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6e382d6d-a79e-4598-a93a-769fcfd16fdc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48412025-03-04T03:21:37.723ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6e382d6d-a79e-4598-a93a-769fcfd16fdc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48422025-03-04T03:21:37.724ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1805 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6e382d6d-a79e-4598-a93a-769fcfd16fdc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48432025-03-04T03:21:37.769ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
48442025-03-04T03:21:37.769ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
48452025-03-04T03:21:37.799ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
48462025-03-04T03:21:37.799ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:47255
48472025-03-04T03:21:37.799ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47255 req_id = 760b4f87-b25d-4857-ad1e-2498f76102aa roles = RoleSet { roles: {} } uri = /metrics/producers
48482025-03-04T03:21:37.800ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47255 req_id = 760b4f87-b25d-4857-ad1e-2498f76102aa resource = Database result = Ok(()) uri = /metrics/producers
48492025-03-04T03:21:37.835ZINFOnexus: assigned collector to new producer collector_id = ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a file = nexus/src/app/oximeter.rs:76 producer_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
48502025-03-04T03:21:37.835ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 36077 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47255 req_id = 760b4f87-b25d-4857-ad1e-2498f76102aa response_code = 201 uri = /metrics/producers
48512025-03-04T03:21:37.835ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "760b4f87-b25d-4857-ad1e-2498f76102aa", "content-length": "41", "date": "Tue, 04 Mar 2025 03:21:37 GMT"} })
48522025-03-04T03:21:37.837ZDEBGnexus (producer-registration-task): registered with nexus successfully
48532025-03-04T03:21:37.837ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
48542025-03-04T03:21:37.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 475 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = ac312441-d1a6-434b-996f-dde09e832762 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
48552025-03-04T03:21:38.743ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be3d1eb1-7cb8-4142-84dd-f5b948f75293 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48562025-03-04T03:21:38.743ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be3d1eb1-7cb8-4142-84dd-f5b948f75293 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48572025-03-04T03:21:38.745ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1732 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be3d1eb1-7cb8-4142-84dd-f5b948f75293 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48582025-03-04T03:21:39.764ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a35a6f3a-9ab1-4473-9d84-0ac97eb7708d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48592025-03-04T03:21:39.764ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a35a6f3a-9ab1-4473-9d84-0ac97eb7708d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48602025-03-04T03:21:39.766ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2387 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a35a6f3a-9ab1-4473-9d84-0ac97eb7708d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48612025-03-04T03:21:40.785ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c98f4aeb-ac45-4d1d-bf26-1dad852edeec roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48622025-03-04T03:21:40.785ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c98f4aeb-ac45-4d1d-bf26-1dad852edeec resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48632025-03-04T03:21:40.787ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2099 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c98f4aeb-ac45-4d1d-bf26-1dad852edeec response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48642025-03-04T03:21:41.806ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fea9cfb5-edc1-4f5e-8f7d-cc05a5148775 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48652025-03-04T03:21:41.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fea9cfb5-edc1-4f5e-8f7d-cc05a5148775 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48662025-03-04T03:21:41.808ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2077 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fea9cfb5-edc1-4f5e-8f7d-cc05a5148775 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48672025-03-04T03:21:42.828ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0427ad80-6aa2-4e91-804f-ded459fcf0a6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48682025-03-04T03:21:42.829ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0427ad80-6aa2-4e91-804f-ded459fcf0a6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48692025-03-04T03:21:42.831ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2674 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0427ad80-6aa2-4e91-804f-ded459fcf0a6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48702025-03-04T03:21:43.851ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c3613d0a-1250-45c3-9f23-5d2467a97b91 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48712025-03-04T03:21:43.851ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c3613d0a-1250-45c3-9f23-5d2467a97b91 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48722025-03-04T03:21:43.853ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2497 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c3613d0a-1250-45c3-9f23-5d2467a97b91 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48732025-03-04T03:21:44.872ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de3cff02-4549-4fc2-966b-6dda863a89ab roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48742025-03-04T03:21:44.872ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de3cff02-4549-4fc2-966b-6dda863a89ab resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48752025-03-04T03:21:44.874ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de3cff02-4549-4fc2-966b-6dda863a89ab response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48762025-03-04T03:21:45.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d193c8a-28ca-427c-ac2e-6d06af94d54a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48772025-03-04T03:21:45.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d193c8a-28ca-427c-ac2e-6d06af94d54a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48782025-03-04T03:21:45.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1971 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d193c8a-28ca-427c-ac2e-6d06af94d54a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48792025-03-04T03:21:46.914ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c4b19857-1b20-4057-b1de-ca5ddc08aeea roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48802025-03-04T03:21:46.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c4b19857-1b20-4057-b1de-ca5ddc08aeea resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48812025-03-04T03:21:46.916ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2163 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c4b19857-1b20-4057-b1de-ca5ddc08aeea response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
48822025-03-04T03:21:47.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
48832025-03-04T03:21:47.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
48842025-03-04T03:21:47.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48852025-03-04T03:21:47.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48862025-03-04T03:21:47.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48872025-03-04T03:21:47.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48882025-03-04T03:21:47.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48892025-03-04T03:21:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48902025-03-04T03:21:47.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48912025-03-04T03:21:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48922025-03-04T03:21:47.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48932025-03-04T03:21:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48942025-03-04T03:21:47.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48952025-03-04T03:21:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48962025-03-04T03:21:47.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48972025-03-04T03:21:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
48982025-03-04T03:21:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48992025-03-04T03:21:47.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49002025-03-04T03:21:47.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
49012025-03-04T03:21:47.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.8571ms iteration = 23 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
49022025-03-04T03:21:47.237ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.230668ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49032025-03-04T03:21:47.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 554 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 52fac1ed-a23c-4f51-b9f3-3ebc2cc52bd7 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
49042025-03-04T03:21:47.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 20ec6beb-2010-45dd-831b-dc8fb270589a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49052025-03-04T03:21:47.936ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 20ec6beb-2010-45dd-831b-dc8fb270589a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49062025-03-04T03:21:47.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1762 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 20ec6beb-2010-45dd-831b-dc8fb270589a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49072025-03-04T03:21:48.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c3c4738-4d74-4b21-bc84-d479eca89391 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49082025-03-04T03:21:48.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c3c4738-4d74-4b21-bc84-d479eca89391 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49092025-03-04T03:21:48.959ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2630 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c3c4738-4d74-4b21-bc84-d479eca89391 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49102025-03-04T03:21:49.979ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c709e24a-d354-456b-897f-766c7e11ad83 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49112025-03-04T03:21:49.979ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c709e24a-d354-456b-897f-766c7e11ad83 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49122025-03-04T03:21:49.981ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1851 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c709e24a-d354-456b-897f-766c7e11ad83 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49132025-03-04T03:21:51.000ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 71502e47-8738-48f6-8cfc-4fb8a7aa992e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49142025-03-04T03:21:51.000ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 71502e47-8738-48f6-8cfc-4fb8a7aa992e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49152025-03-04T03:21:51.002ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2682 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 71502e47-8738-48f6-8cfc-4fb8a7aa992e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49162025-03-04T03:21:52.021ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0230a0f6-5e7a-432d-8e0a-7bed812b3263 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49172025-03-04T03:21:52.022ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0230a0f6-5e7a-432d-8e0a-7bed812b3263 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49182025-03-04T03:21:52.024ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2832 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0230a0f6-5e7a-432d-8e0a-7bed812b3263 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49192025-03-04T03:21:53.043ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9693e13-c73d-46d3-b00d-983c14947515 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49202025-03-04T03:21:53.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9693e13-c73d-46d3-b00d-983c14947515 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49212025-03-04T03:21:53.045ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1985 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9693e13-c73d-46d3-b00d-983c14947515 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49222025-03-04T03:21:54.064ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c63a148-4f7c-4083-8b52-d278492727a6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49232025-03-04T03:21:54.064ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c63a148-4f7c-4083-8b52-d278492727a6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49242025-03-04T03:21:54.066ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2320 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c63a148-4f7c-4083-8b52-d278492727a6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49252025-03-04T03:21:55.086ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e03c81ba-7a6a-4320-ade7-94171eccce2b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49262025-03-04T03:21:55.087ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e03c81ba-7a6a-4320-ade7-94171eccce2b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49272025-03-04T03:21:55.089ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2706 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e03c81ba-7a6a-4320-ade7-94171eccce2b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49282025-03-04T03:21:56.108ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dfa3cbfd-bba3-4cc7-a7ed-9ab1fa70ae1b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49292025-03-04T03:21:56.109ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dfa3cbfd-bba3-4cc7-a7ed-9ab1fa70ae1b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49302025-03-04T03:21:56.111ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2508 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dfa3cbfd-bba3-4cc7-a7ed-9ab1fa70ae1b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49312025-03-04T03:21:57.130ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fc9d1af0-45e8-4aa6-8d38-6c8ac6d50e55 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49322025-03-04T03:21:57.130ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fc9d1af0-45e8-4aa6-8d38-6c8ac6d50e55 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49332025-03-04T03:21:57.132ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2003 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fc9d1af0-45e8-4aa6-8d38-6c8ac6d50e55 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49342025-03-04T03:21:57.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
49352025-03-04T03:21:57.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49362025-03-04T03:21:57.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49372025-03-04T03:21:57.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
49382025-03-04T03:21:57.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49392025-03-04T03:21:57.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49402025-03-04T03:21:57.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49412025-03-04T03:21:57.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49422025-03-04T03:21:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49432025-03-04T03:21:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49442025-03-04T03:21:57.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49452025-03-04T03:21:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49462025-03-04T03:21:57.208ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.653995ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49472025-03-04T03:21:57.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49482025-03-04T03:21:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49492025-03-04T03:21:57.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49502025-03-04T03:21:57.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
49512025-03-04T03:21:57.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49522025-03-04T03:21:57.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49532025-03-04T03:21:57.222ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
49542025-03-04T03:21:57.222ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.73625ms iteration = 24 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
49552025-03-04T03:21:57.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 540 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = ef4fac41-7238-46bb-a892-f1b3024bf239 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
49562025-03-04T03:21:58.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f08ff94d-607a-42b6-bccd-699c3378dd95 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49572025-03-04T03:21:58.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f08ff94d-607a-42b6-bccd-699c3378dd95 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49582025-03-04T03:21:58.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2508 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f08ff94d-607a-42b6-bccd-699c3378dd95 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49592025-03-04T03:21:59.172ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7615372-201e-409d-a07c-e83a4f96646e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49602025-03-04T03:21:59.172ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7615372-201e-409d-a07c-e83a4f96646e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49612025-03-04T03:21:59.174ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2126 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a7615372-201e-409d-a07c-e83a4f96646e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49622025-03-04T03:22:00.193ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b35b4019-d1e2-4305-bdaa-cfa66db8584a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49632025-03-04T03:22:00.194ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b35b4019-d1e2-4305-bdaa-cfa66db8584a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49642025-03-04T03:22:00.196ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2372 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b35b4019-d1e2-4305-bdaa-cfa66db8584a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49652025-03-04T03:22:01.214ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d34ad11-4196-4e41-a196-fda565e0eb7c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49662025-03-04T03:22:01.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d34ad11-4196-4e41-a196-fda565e0eb7c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49672025-03-04T03:22:01.217ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2312 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d34ad11-4196-4e41-a196-fda565e0eb7c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49682025-03-04T03:22:02.236ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af86ff4d-09cb-4e72-838e-fa81eb6a7e81 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49692025-03-04T03:22:02.236ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af86ff4d-09cb-4e72-838e-fa81eb6a7e81 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49702025-03-04T03:22:02.238ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1656 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af86ff4d-09cb-4e72-838e-fa81eb6a7e81 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49712025-03-04T03:22:03.256ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 46abfec1-1b34-418c-b946-fde2e353ce7e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49722025-03-04T03:22:03.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 46abfec1-1b34-418c-b946-fde2e353ce7e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49732025-03-04T03:22:03.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2372 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 46abfec1-1b34-418c-b946-fde2e353ce7e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49742025-03-04T03:22:04.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04d54fd0-a5d2-4b9f-9d88-b31b9304ea1f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49752025-03-04T03:22:04.278ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04d54fd0-a5d2-4b9f-9d88-b31b9304ea1f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49762025-03-04T03:22:04.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2496 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04d54fd0-a5d2-4b9f-9d88-b31b9304ea1f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49772025-03-04T03:22:05.300ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6ec6670-1e5a-4ca9-94f5-5401c708a668 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49782025-03-04T03:22:05.300ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6ec6670-1e5a-4ca9-94f5-5401c708a668 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49792025-03-04T03:22:05.302ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2160 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6ec6670-1e5a-4ca9-94f5-5401c708a668 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49802025-03-04T03:22:06.321ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9742c4b6-96bc-4106-8a4b-4359fa572a40 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49812025-03-04T03:22:06.322ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9742c4b6-96bc-4106-8a4b-4359fa572a40 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49822025-03-04T03:22:06.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2207 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9742c4b6-96bc-4106-8a4b-4359fa572a40 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
49832025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
49842025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
49852025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
49862025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
49872025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
49882025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
49892025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
49902025-03-04T03:22:07.202ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:12:07.202492418 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
49912025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
49922025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
49932025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
49942025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
49952025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
49962025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
49972025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
49982025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
49992025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
50002025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
50012025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
50022025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
50032025-03-04T03:22:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
50042025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
50052025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
50062025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
50072025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
50082025-03-04T03:22:07.203ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1239, tv_nsec: 400462152 } })
50092025-03-04T03:22:07.203ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1239, tv_nsec: 400513512 } })
50102025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
50112025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
50122025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
50132025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50142025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
50152025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
50162025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50172025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 9 reason = Timeout
50182025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50192025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
50202025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50212025-03-04T03:22:07.203ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
50222025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.09\u{b5}s iteration = 6 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
50232025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
50242025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
50252025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
50262025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50272025-03-04T03:22:07.203ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
50282025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50292025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
50302025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
50312025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
50322025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50332025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50342025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50352025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50362025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
50372025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50382025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
50392025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.257515ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
50402025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
50412025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
50422025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.244415ms iteration = 5 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
50432025-03-04T03:22:07.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
50442025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
50452025-03-04T03:22:07.204ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:12:07.202492418 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
50462025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
50472025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.252408ms iteration = 5 status = {"expiration":"2025-03-04T03:12:07.202492418Z","pruned":[]}
50482025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50492025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
50502025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
50512025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
50522025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
50532025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
50542025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
50552025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50562025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
50572025-03-04T03:22:07.204ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
50582025-03-04T03:22:07.204ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50592025-03-04T03:22:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50602025-03-04T03:22:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50612025-03-04T03:22:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
50622025-03-04T03:22:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
50632025-03-04T03:22:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
50642025-03-04T03:22:07.205ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50652025-03-04T03:22:07.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
50662025-03-04T03:22:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
50672025-03-04T03:22:07.206ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
50682025-03-04T03:22:07.206ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
50692025-03-04T03:22:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
50702025-03-04T03:22:07.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1239, tv_nsec: 403959294 } })
50712025-03-04T03:22:07.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
50722025-03-04T03:22:07.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
50732025-03-04T03:22:07.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
50742025-03-04T03:22:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
50752025-03-04T03:22:07.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1239, tv_nsec: 404338079 } })
50762025-03-04T03:22:07.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
50772025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
50782025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 9 reason = Timeout
50792025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
50802025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
50812025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
50822025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
50832025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
50842025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
50852025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
50862025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
50872025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
50882025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
50892025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
50902025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.441067ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
50912025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
50922025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
50932025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
50942025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
50952025-03-04T03:22:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
50962025-03-04T03:22:07.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50972025-03-04T03:22:07.209ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50982025-03-04T03:22:07.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50992025-03-04T03:22:07.210ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51002025-03-04T03:22:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
51012025-03-04T03:22:07.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51022025-03-04T03:22:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51032025-03-04T03:22:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
51042025-03-04T03:22:07.212ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
51052025-03-04T03:22:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
51062025-03-04T03:22:07.212ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
51072025-03-04T03:22:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51082025-03-04T03:22:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
51092025-03-04T03:22:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51102025-03-04T03:22:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
51112025-03-04T03:22:07.216ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51122025-03-04T03:22:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51132025-03-04T03:22:07.217ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51142025-03-04T03:22:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51152025-03-04T03:22:07.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51162025-03-04T03:22:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51172025-03-04T03:22:07.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51182025-03-04T03:22:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
51192025-03-04T03:22:07.220ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51202025-03-04T03:22:07.221ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51212025-03-04T03:22:07.221ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
51222025-03-04T03:22:07.221ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51232025-03-04T03:22:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51242025-03-04T03:22:07.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
51252025-03-04T03:22:07.221ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51262025-03-04T03:22:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51272025-03-04T03:22:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
51282025-03-04T03:22:07.225ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
51292025-03-04T03:22:07.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
51302025-03-04T03:22:07.226ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 17.739797ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
51312025-03-04T03:22:07.226ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.786798ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
51322025-03-04T03:22:07.228ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 19.757513ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
51332025-03-04T03:22:07.229ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1239, tv_nsec: 427211170 } })
51342025-03-04T03:22:07.230ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51352025-03-04T03:22:07.230ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
51362025-03-04T03:22:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51372025-03-04T03:22:07.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51382025-03-04T03:22:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51392025-03-04T03:22:07.232ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51402025-03-04T03:22:07.232ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
51412025-03-04T03:22:07.232ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51422025-03-04T03:22:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51432025-03-04T03:22:07.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51442025-03-04T03:22:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
51452025-03-04T03:22:07.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51462025-03-04T03:22:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51472025-03-04T03:22:07.262ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
51482025-03-04T03:22:07.263ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
51492025-03-04T03:22:07.263ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 56.637884ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
51502025-03-04T03:22:07.263ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
51512025-03-04T03:22:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
51522025-03-04T03:22:07.265ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51532025-03-04T03:22:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51542025-03-04T03:22:07.266ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
51552025-03-04T03:22:07.266ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
51562025-03-04T03:22:07.266ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
51572025-03-04T03:22:07.266ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51582025-03-04T03:22:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
51592025-03-04T03:22:07.268ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 59.578871ms iteration = 9 status = {"errors":[],"requests_created_ok":[]}
51602025-03-04T03:22:07.268ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51612025-03-04T03:22:07.268ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51622025-03-04T03:22:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51632025-03-04T03:22:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51642025-03-04T03:22:07.270ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.839177ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51652025-03-04T03:22:07.271ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51662025-03-04T03:22:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
51672025-03-04T03:22:07.273ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
51682025-03-04T03:22:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
51692025-03-04T03:22:07.274ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51702025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
51712025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51722025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
51732025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
51742025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
51752025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
51762025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51772025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51782025-03-04T03:22:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
51792025-03-04T03:22:07.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
51802025-03-04T03:22:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
51812025-03-04T03:22:07.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51822025-03-04T03:22:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51832025-03-04T03:22:07.279ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
51842025-03-04T03:22:07.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51852025-03-04T03:22:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51862025-03-04T03:22:07.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
51872025-03-04T03:22:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51882025-03-04T03:22:07.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51892025-03-04T03:22:07.286ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 78.428591ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
51902025-03-04T03:22:07.287ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
51912025-03-04T03:22:07.287ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
51922025-03-04T03:22:07.287ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 83.888778ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
51932025-03-04T03:22:07.287ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51942025-03-04T03:22:07.287ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
51952025-03-04T03:22:07.287ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 84.147181ms iteration = 25 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
51962025-03-04T03:22:07.287ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51972025-03-04T03:22:07.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51982025-03-04T03:22:07.289ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51992025-03-04T03:22:07.289ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
52002025-03-04T03:22:07.289ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 84.327954ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
52012025-03-04T03:22:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52022025-03-04T03:22:07.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52032025-03-04T03:22:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
52042025-03-04T03:22:07.290ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52052025-03-04T03:22:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52062025-03-04T03:22:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52072025-03-04T03:22:07.292ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52082025-03-04T03:22:07.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52092025-03-04T03:22:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
52102025-03-04T03:22:07.294ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52112025-03-04T03:22:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52122025-03-04T03:22:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
52132025-03-04T03:22:07.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52142025-03-04T03:22:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52152025-03-04T03:22:07.307ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 104.534971ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
52162025-03-04T03:22:07.308ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52172025-03-04T03:22:07.308ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52182025-03-04T03:22:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
52192025-03-04T03:22:07.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52202025-03-04T03:22:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52212025-03-04T03:22:07.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
52222025-03-04T03:22:07.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52232025-03-04T03:22:07.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52242025-03-04T03:22:07.313ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 109.907887ms iteration = 9 status = 0
52252025-03-04T03:22:07.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
52262025-03-04T03:22:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
52272025-03-04T03:22:07.319ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
52282025-03-04T03:22:07.319ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 114.538994ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
52292025-03-04T03:22:07.326ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
52302025-03-04T03:22:07.326ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
52312025-03-04T03:22:07.326ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52322025-03-04T03:22:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52332025-03-04T03:22:07.328ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52342025-03-04T03:22:07.330ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
52352025-03-04T03:22:07.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52362025-03-04T03:22:07.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52372025-03-04T03:22:07.331ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52382025-03-04T03:22:07.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52392025-03-04T03:22:07.331ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
52402025-03-04T03:22:07.331ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 128.544116ms iteration = 9 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
52412025-03-04T03:22:07.332ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52422025-03-04T03:22:07.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52432025-03-04T03:22:07.334ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52442025-03-04T03:22:07.334ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
52452025-03-04T03:22:07.334ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
52462025-03-04T03:22:07.334ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
52472025-03-04T03:22:07.334ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 132.374242ms iteration = 5 status = {"generation":1}
52482025-03-04T03:22:07.335ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
52492025-03-04T03:22:07.335ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52502025-03-04T03:22:07.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52512025-03-04T03:22:07.343ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6e3b51d-e571-40c1-8fd8-dcedb687289a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
52522025-03-04T03:22:07.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6e3b51d-e571-40c1-8fd8-dcedb687289a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
52532025-03-04T03:22:07.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2129 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6e3b51d-e571-40c1-8fd8-dcedb687289a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
52542025-03-04T03:22:07.345ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
52552025-03-04T03:22:07.345ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
52562025-03-04T03:22:07.345ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8bf978d-d2ec-45fe-b0f6-7245493f5d03", "content-length": "26", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
52572025-03-04T03:22:07.345ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
52582025-03-04T03:22:07.347ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
52592025-03-04T03:22:07.348ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
52602025-03-04T03:22:07.348ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
52612025-03-04T03:22:07.348ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52622025-03-04T03:22:07.348ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5ffca08-2488-4929-a546-99019b08b74e", "content-length": "26", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
52632025-03-04T03:22:07.348ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
52642025-03-04T03:22:07.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52652025-03-04T03:22:07.350ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
52662025-03-04T03:22:07.354ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
52672025-03-04T03:22:07.356ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
52682025-03-04T03:22:07.356ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
52692025-03-04T03:22:07.356ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52702025-03-04T03:22:07.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52712025-03-04T03:22:07.358ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52722025-03-04T03:22:07.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52732025-03-04T03:22:07.362ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
52742025-03-04T03:22:07.362ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52752025-03-04T03:22:07.363ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52762025-03-04T03:22:07.365ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
52772025-03-04T03:22:07.365ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52782025-03-04T03:22:07.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52792025-03-04T03:22:07.366ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
52802025-03-04T03:22:07.366ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52812025-03-04T03:22:07.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52822025-03-04T03:22:07.367ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52832025-03-04T03:22:07.368ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
52842025-03-04T03:22:07.368ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
52852025-03-04T03:22:07.368ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
52862025-03-04T03:22:07.368ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 166.321608ms iteration = 5 status = {"generation":2}
52872025-03-04T03:22:07.369ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
52882025-03-04T03:22:07.369ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52892025-03-04T03:22:07.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52902025-03-04T03:22:07.378ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 169.595158ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
52912025-03-04T03:22:07.382ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
52922025-03-04T03:22:07.382ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 179.951256ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
52932025-03-04T03:22:07.392ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
52942025-03-04T03:22:07.392ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
52952025-03-04T03:22:07.392ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "745e0c7b-8c04-4482-890c-6c72a8be77e7", "content-length": "26", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
52962025-03-04T03:22:07.392ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
52972025-03-04T03:22:07.393ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
52982025-03-04T03:22:07.393ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 188.672702ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
52992025-03-04T03:22:07.393ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
53002025-03-04T03:22:07.416ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 213.457426ms iteration = 11 status = {"physical_disks_added":0}
53012025-03-04T03:22:07.494ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 291.736146ms iteration = 9 status = {}
53022025-03-04T03:22:07.517ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
53032025-03-04T03:22:07.517ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "86267f68-50c7-4ebb-85d9-98f668bab319", "content-length": "1", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53042025-03-04T03:22:07.518ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
53052025-03-04T03:22:07.518ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "fa3abec4-20c6-4e09-8b84-cb03049739ec", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53062025-03-04T03:22:07.518ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53072025-03-04T03:22:07.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
53082025-03-04T03:22:07.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
53092025-03-04T03:22:07.521ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "8200a426-9246-48c2-9610-bff0cc9447d5", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53102025-03-04T03:22:07.521ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53112025-03-04T03:22:07.522ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 320.415187ms iteration = 9 status = null
53122025-03-04T03:22:07.523ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53132025-03-04T03:22:07.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 320.455207ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
53142025-03-04T03:22:07.533ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
53152025-03-04T03:22:07.633ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
53162025-03-04T03:22:07.633ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
53172025-03-04T03:22:07.634ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc480c39-5197-43d1-a80f-a720d4605968", "content-length": "372", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53182025-03-04T03:22:07.634ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53192025-03-04T03:22:07.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53202025-03-04T03:22:07.638ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
53212025-03-04T03:22:07.638ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "b0a5fd07-445b-480e-8c31-01e258e7ac9d", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53222025-03-04T03:22:07.638ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53232025-03-04T03:22:07.639ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "7bebfeab-d56e-4c31-a65b-af95bf8e538e", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53242025-03-04T03:22:07.639ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53252025-03-04T03:22:07.640ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1181da42-4efc-4dce-92e1-570d78ea956f", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53262025-03-04T03:22:07.640ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53272025-03-04T03:22:07.642ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53282025-03-04T03:22:07.642ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 439.77345ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
53292025-03-04T03:22:07.652ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
53302025-03-04T03:22:07.652ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e2e5fbb-0ed4-4167-bd33-dcb8d1b2a246", "content-length": "691", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53312025-03-04T03:22:07.653ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
53322025-03-04T03:22:07.653ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
53332025-03-04T03:22:07.653ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
53342025-03-04T03:22:07.655ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 452.353594ms iteration = 9 status = {}
53352025-03-04T03:22:07.678ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
53362025-03-04T03:22:07.678ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
53372025-03-04T03:22:07.678ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ce7a2b7e-63e7-4e4d-8556-ec127ffab6ed", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53382025-03-04T03:22:07.678ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53392025-03-04T03:22:07.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53402025-03-04T03:22:07.680ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53412025-03-04T03:22:07.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53422025-03-04T03:22:07.683ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53432025-03-04T03:22:07.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53442025-03-04T03:22:07.683ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53452025-03-04T03:22:07.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53462025-03-04T03:22:07.685ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
53472025-03-04T03:22:07.685ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53482025-03-04T03:22:07.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53492025-03-04T03:22:07.687ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
53502025-03-04T03:22:07.687ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
53512025-03-04T03:22:07.687ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "153c5723-2f19-4d98-a5fd-66ff6b4a12a8", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53522025-03-04T03:22:07.689ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 484.518569ms iteration = 9 status = {}
53532025-03-04T03:22:07.695ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
53542025-03-04T03:22:07.696ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab451632-2536-4f41-872f-a457496f1039", "content-length": "134", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53552025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
53562025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
53572025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
53582025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
53592025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
53602025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
53612025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
53622025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
53632025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
53642025-03-04T03:22:07.696ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
53652025-03-04T03:22:07.697ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
53662025-03-04T03:22:07.697ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53672025-03-04T03:22:07.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53682025-03-04T03:22:07.699ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
53692025-03-04T03:22:07.699ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "369c3acf-70d7-4323-a618-616e90cb056a", "content-length": "2", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53702025-03-04T03:22:07.699ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
53712025-03-04T03:22:07.699ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e94f260b-ce1a-4563-ae28-4383ebe40a61", "content-length": "51", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53722025-03-04T03:22:07.699ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
53732025-03-04T03:22:07.699ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
53742025-03-04T03:22:07.789ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "bea7a11f-75fd-426f-89f0-c1aa0024fffc", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53752025-03-04T03:22:07.790ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
53762025-03-04T03:22:07.790ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
53772025-03-04T03:22:07.790ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "adfe59af-fb69-4b47-a87f-1803968a476e", "content-length": "682", "date": "Tue, 04 Mar 2025 03:22:07 GMT"} })
53782025-03-04T03:22:07.790ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53792025-03-04T03:22:07.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53802025-03-04T03:22:07.794ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53812025-03-04T03:22:07.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53822025-03-04T03:22:07.798ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53832025-03-04T03:22:07.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53842025-03-04T03:22:07.800ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53852025-03-04T03:22:07.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53862025-03-04T03:22:07.808ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 605.175277ms iteration = 9 status = {}
53872025-03-04T03:22:07.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 481 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = ebf25b6f-f367-4898-bfbb-e99a9176c60c response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
53882025-03-04T03:22:08.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 79cfe1fe-ac59-45af-9c10-105eaf18ab33 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53892025-03-04T03:22:08.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 79cfe1fe-ac59-45af-9c10-105eaf18ab33 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53902025-03-04T03:22:08.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2501 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 79cfe1fe-ac59-45af-9c10-105eaf18ab33 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53912025-03-04T03:22:09.387ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb84c76d-42ca-45de-93ab-cd6e959c7aba roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53922025-03-04T03:22:09.388ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb84c76d-42ca-45de-93ab-cd6e959c7aba resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53932025-03-04T03:22:09.390ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2555 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb84c76d-42ca-45de-93ab-cd6e959c7aba response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53942025-03-04T03:22:10.408ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b47d158f-5ec7-4625-a335-9e2f107089c7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53952025-03-04T03:22:10.408ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b47d158f-5ec7-4625-a335-9e2f107089c7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53962025-03-04T03:22:10.411ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2910 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b47d158f-5ec7-4625-a335-9e2f107089c7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53972025-03-04T03:22:11.431ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 333a8dd5-b2c8-433e-b1ec-cf8e34eb76fd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53982025-03-04T03:22:11.432ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 333a8dd5-b2c8-433e-b1ec-cf8e34eb76fd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
53992025-03-04T03:22:11.434ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2380 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 333a8dd5-b2c8-433e-b1ec-cf8e34eb76fd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54002025-03-04T03:22:12.453ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a48dee2c-5b5d-4433-9bc2-36dd6c45b5df roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54012025-03-04T03:22:12.454ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a48dee2c-5b5d-4433-9bc2-36dd6c45b5df resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54022025-03-04T03:22:12.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2932 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a48dee2c-5b5d-4433-9bc2-36dd6c45b5df response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54032025-03-04T03:22:13.474ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0cc842b8-0fea-40c4-9b1b-f85f564dca76 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54042025-03-04T03:22:13.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0cc842b8-0fea-40c4-9b1b-f85f564dca76 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54052025-03-04T03:22:13.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2097 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0cc842b8-0fea-40c4-9b1b-f85f564dca76 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54062025-03-04T03:22:14.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2dd1b98b-18cb-42b7-946a-d950ec75a55f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54072025-03-04T03:22:14.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2dd1b98b-18cb-42b7-946a-d950ec75a55f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54082025-03-04T03:22:14.496ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1981 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2dd1b98b-18cb-42b7-946a-d950ec75a55f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54092025-03-04T03:22:15.516ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab816e06-de3b-44b2-8c83-68c075bb17e5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54102025-03-04T03:22:15.517ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab816e06-de3b-44b2-8c83-68c075bb17e5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54112025-03-04T03:22:15.519ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2528 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab816e06-de3b-44b2-8c83-68c075bb17e5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54122025-03-04T03:22:16.538ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5f847e1-4ae3-4eee-bc38-baaea1cb567b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54132025-03-04T03:22:16.539ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5f847e1-4ae3-4eee-bc38-baaea1cb567b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54142025-03-04T03:22:16.540ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2237 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5f847e1-4ae3-4eee-bc38-baaea1cb567b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54152025-03-04T03:22:17.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
54162025-03-04T03:22:17.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54172025-03-04T03:22:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54182025-03-04T03:22:17.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
54192025-03-04T03:22:17.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54202025-03-04T03:22:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54212025-03-04T03:22:17.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54222025-03-04T03:22:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54232025-03-04T03:22:17.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54242025-03-04T03:22:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54252025-03-04T03:22:17.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54262025-03-04T03:22:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54272025-03-04T03:22:17.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54282025-03-04T03:22:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
54292025-03-04T03:22:17.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54302025-03-04T03:22:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54312025-03-04T03:22:17.223ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
54322025-03-04T03:22:17.223ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.190798ms iteration = 26 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
54332025-03-04T03:22:17.234ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54342025-03-04T03:22:17.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54352025-03-04T03:22:17.263ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.75715ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54362025-03-04T03:22:17.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bd72aabb-5a31-4a35-a05f-b611db7db50b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54372025-03-04T03:22:17.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bd72aabb-5a31-4a35-a05f-b611db7db50b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54382025-03-04T03:22:17.562ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1981 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bd72aabb-5a31-4a35-a05f-b611db7db50b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54392025-03-04T03:22:17.894ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 485 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 66f0019e-beb2-43da-946a-0a4601907d29 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
54402025-03-04T03:22:18.582ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 693a5d47-fe72-4f1e-b79e-35338f52ac49 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54412025-03-04T03:22:18.583ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 693a5d47-fe72-4f1e-b79e-35338f52ac49 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54422025-03-04T03:22:18.584ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2104 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 693a5d47-fe72-4f1e-b79e-35338f52ac49 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54432025-03-04T03:22:19.604ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 482725ca-a533-4e8a-b9d8-0cee9f64536a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54442025-03-04T03:22:19.605ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 482725ca-a533-4e8a-b9d8-0cee9f64536a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54452025-03-04T03:22:19.607ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2482 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 482725ca-a533-4e8a-b9d8-0cee9f64536a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54462025-03-04T03:22:20.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eca5ff11-2466-4263-a678-a62284da0f30 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54472025-03-04T03:22:20.626ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eca5ff11-2466-4263-a678-a62284da0f30 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54482025-03-04T03:22:20.627ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2001 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eca5ff11-2466-4263-a678-a62284da0f30 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54492025-03-04T03:22:21.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c15c27b6-5634-441d-bc34-5ba4d14a3a6d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54502025-03-04T03:22:21.648ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c15c27b6-5634-441d-bc34-5ba4d14a3a6d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54512025-03-04T03:22:21.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2310 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c15c27b6-5634-441d-bc34-5ba4d14a3a6d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54522025-03-04T03:22:22.669ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5f306cf4-d4b6-4312-95d0-feaa09670966 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54532025-03-04T03:22:22.669ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5f306cf4-d4b6-4312-95d0-feaa09670966 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54542025-03-04T03:22:22.671ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2504 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5f306cf4-d4b6-4312-95d0-feaa09670966 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54552025-03-04T03:22:23.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7d8ac8e3-3c70-415d-957c-4e1b64f3c26f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54562025-03-04T03:22:23.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7d8ac8e3-3c70-415d-957c-4e1b64f3c26f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54572025-03-04T03:22:23.695ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2431 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7d8ac8e3-3c70-415d-957c-4e1b64f3c26f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54582025-03-04T03:22:24.713ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7362b28-5ab6-4304-9800-d34c7177063c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54592025-03-04T03:22:24.714ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7362b28-5ab6-4304-9800-d34c7177063c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54602025-03-04T03:22:24.715ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1934 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7362b28-5ab6-4304-9800-d34c7177063c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54612025-03-04T03:22:25.734ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e43387c4-5048-4b40-a060-c854d9f1f492 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54622025-03-04T03:22:25.734ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e43387c4-5048-4b40-a060-c854d9f1f492 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54632025-03-04T03:22:25.736ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2122 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e43387c4-5048-4b40-a060-c854d9f1f492 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54642025-03-04T03:22:26.755ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d0b5a123-bf2e-4903-a838-f195b4ccecfc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54652025-03-04T03:22:26.755ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d0b5a123-bf2e-4903-a838-f195b4ccecfc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54662025-03-04T03:22:26.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4939 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d0b5a123-bf2e-4903-a838-f195b4ccecfc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54672025-03-04T03:22:27.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
54682025-03-04T03:22:27.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54692025-03-04T03:22:27.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54702025-03-04T03:22:27.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
54712025-03-04T03:22:27.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54722025-03-04T03:22:27.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54732025-03-04T03:22:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54742025-03-04T03:22:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54752025-03-04T03:22:27.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54762025-03-04T03:22:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54772025-03-04T03:22:27.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54782025-03-04T03:22:27.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54792025-03-04T03:22:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54802025-03-04T03:22:27.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
54812025-03-04T03:22:27.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54822025-03-04T03:22:27.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54832025-03-04T03:22:27.225ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
54842025-03-04T03:22:27.225ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.885908ms iteration = 27 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
54852025-03-04T03:22:27.234ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54862025-03-04T03:22:27.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54872025-03-04T03:22:27.235ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.356594ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54882025-03-04T03:22:27.779ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4234a8e1-5fc3-4fb3-be9a-a24a4ffa5c8b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54892025-03-04T03:22:27.779ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4234a8e1-5fc3-4fb3-be9a-a24a4ffa5c8b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54902025-03-04T03:22:27.780ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1811 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4234a8e1-5fc3-4fb3-be9a-a24a4ffa5c8b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54912025-03-04T03:22:27.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 628 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 349459ed-c2b5-4e50-a15a-7f7c6811b8e3 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
54922025-03-04T03:22:28.798ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6a75633-fa9f-4326-9b6e-94c1f6c1badb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54932025-03-04T03:22:28.798ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6a75633-fa9f-4326-9b6e-94c1f6c1badb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54942025-03-04T03:22:28.800ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2146 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6a75633-fa9f-4326-9b6e-94c1f6c1badb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54952025-03-04T03:22:29.819ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 371f3a09-fcc0-4b41-a64c-ca0058007e6b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54962025-03-04T03:22:29.819ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 371f3a09-fcc0-4b41-a64c-ca0058007e6b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54972025-03-04T03:22:29.821ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1732 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 371f3a09-fcc0-4b41-a64c-ca0058007e6b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54982025-03-04T03:22:30.839ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aabe6519-6f24-4387-80cb-35e38de45d64 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
54992025-03-04T03:22:30.840ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aabe6519-6f24-4387-80cb-35e38de45d64 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55002025-03-04T03:22:30.841ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2203 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aabe6519-6f24-4387-80cb-35e38de45d64 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55012025-03-04T03:22:31.860ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e6554c8-431b-4d3c-a5c3-11843e0a5424 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55022025-03-04T03:22:31.861ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e6554c8-431b-4d3c-a5c3-11843e0a5424 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55032025-03-04T03:22:31.863ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2577 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e6554c8-431b-4d3c-a5c3-11843e0a5424 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55042025-03-04T03:22:32.882ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 58001b1a-0368-4a79-b42c-4e9553d4a80e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55052025-03-04T03:22:32.882ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 58001b1a-0368-4a79-b42c-4e9553d4a80e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55062025-03-04T03:22:32.884ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2325 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 58001b1a-0368-4a79-b42c-4e9553d4a80e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55072025-03-04T03:22:32.889ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:41224
55082025-03-04T03:22:32.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41224 req_id = fd75c0bd-b342-45f5-b72d-eb77e2d31099 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
55092025-03-04T03:22:32.889ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41224 req_id = fd75c0bd-b342-45f5-b72d-eb77e2d31099 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
55102025-03-04T03:22:32.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2826 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41224 req_id = fd75c0bd-b342-45f5-b72d-eb77e2d31099 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
55112025-03-04T03:22:32.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41224 req_id = 7480be20-720e-42d7-adb6-7b9758ceddc9 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
55122025-03-04T03:22:32.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41224 req_id = 7480be20-720e-42d7-adb6-7b9758ceddc9 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
55132025-03-04T03:22:32.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2588 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41224 req_id = 7480be20-720e-42d7-adb6-7b9758ceddc9 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
55142025-03-04T03:22:33.910ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fcaf153f-40dc-4cc1-aaf2-98396d90cf8b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55152025-03-04T03:22:33.910ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fcaf153f-40dc-4cc1-aaf2-98396d90cf8b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55162025-03-04T03:22:33.912ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2563 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fcaf153f-40dc-4cc1-aaf2-98396d90cf8b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55172025-03-04T03:22:34.932ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b814e17-216b-475e-b165-95e2c16175ef roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55182025-03-04T03:22:34.932ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b814e17-216b-475e-b165-95e2c16175ef resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55192025-03-04T03:22:34.934ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2057 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b814e17-216b-475e-b165-95e2c16175ef response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55202025-03-04T03:22:35.953ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f65b23a-a796-4ba6-9be6-3709d096fcf4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55212025-03-04T03:22:35.953ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f65b23a-a796-4ba6-9be6-3709d096fcf4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55222025-03-04T03:22:35.955ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2038 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6f65b23a-a796-4ba6-9be6-3709d096fcf4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55232025-03-04T03:22:36.974ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9e661b78-a3de-4c04-8fbf-2e5e04b17644 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55242025-03-04T03:22:36.974ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9e661b78-a3de-4c04-8fbf-2e5e04b17644 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55252025-03-04T03:22:36.978ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4508 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9e661b78-a3de-4c04-8fbf-2e5e04b17644 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
55262025-03-04T03:22:37.202ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
55272025-03-04T03:22:37.202ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
55282025-03-04T03:22:37.202ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
55292025-03-04T03:22:37.202ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
55302025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
55312025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
55322025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
55332025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55342025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
55352025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55362025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
55372025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
55382025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
55392025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
55402025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
55412025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55422025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55432025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
55442025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
55452025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55462025-03-04T03:22:37.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55472025-03-04T03:22:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
55482025-03-04T03:22:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55492025-03-04T03:22:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55502025-03-04T03:22:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55512025-03-04T03:22:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55522025-03-04T03:22:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55532025-03-04T03:22:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55542025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
55552025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
55562025-03-04T03:22:37.205ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
55572025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
55582025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
55592025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55602025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
55612025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55622025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
55632025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55642025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
55652025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
55662025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55672025-03-04T03:22:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55682025-03-04T03:22:37.205ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1269, tv_nsec: 402867047 } })
55692025-03-04T03:22:37.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
55702025-03-04T03:22:37.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1269, tv_nsec: 403552075 } })
55712025-03-04T03:22:37.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55722025-03-04T03:22:37.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
55732025-03-04T03:22:37.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55742025-03-04T03:22:37.208ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.378763ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
55752025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
55762025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55772025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 10 reason = Timeout
55782025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
55792025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
55802025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55812025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
55822025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
55832025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
55842025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
55852025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
55862025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
55872025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
55882025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
55892025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
55902025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
55912025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
55922025-03-04T03:22:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
55932025-03-04T03:22:37.210ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55942025-03-04T03:22:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
55952025-03-04T03:22:37.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55962025-03-04T03:22:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55972025-03-04T03:22:37.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55982025-03-04T03:22:37.213ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55992025-03-04T03:22:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
56002025-03-04T03:22:37.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56012025-03-04T03:22:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
56022025-03-04T03:22:37.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56032025-03-04T03:22:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56042025-03-04T03:22:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56052025-03-04T03:22:37.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56062025-03-04T03:22:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56072025-03-04T03:22:37.215ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
56082025-03-04T03:22:37.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56092025-03-04T03:22:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56102025-03-04T03:22:37.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56112025-03-04T03:22:37.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56122025-03-04T03:22:37.220ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1269, tv_nsec: 418059743 } })
56132025-03-04T03:22:37.221ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
56142025-03-04T03:22:37.243ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56152025-03-04T03:22:37.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
56162025-03-04T03:22:37.245ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56172025-03-04T03:22:37.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56182025-03-04T03:22:37.245ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56192025-03-04T03:22:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56202025-03-04T03:22:37.246ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56212025-03-04T03:22:37.246ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56222025-03-04T03:22:37.246ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
56232025-03-04T03:22:37.246ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
56242025-03-04T03:22:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
56252025-03-04T03:22:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56262025-03-04T03:22:37.247ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.636543ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56272025-03-04T03:22:37.248ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56282025-03-04T03:22:37.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56292025-03-04T03:22:37.249ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56302025-03-04T03:22:37.249ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56312025-03-04T03:22:37.249ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56322025-03-04T03:22:37.249ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
56332025-03-04T03:22:37.249ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
56342025-03-04T03:22:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56352025-03-04T03:22:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
56362025-03-04T03:22:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
56372025-03-04T03:22:37.252ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56382025-03-04T03:22:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56392025-03-04T03:22:37.255ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56402025-03-04T03:22:37.255ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56412025-03-04T03:22:37.256ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
56422025-03-04T03:22:37.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
56432025-03-04T03:22:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
56442025-03-04T03:22:37.258ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
56452025-03-04T03:22:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
56462025-03-04T03:22:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
56472025-03-04T03:22:37.258ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56482025-03-04T03:22:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56492025-03-04T03:22:37.260ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 50.967895ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
56502025-03-04T03:22:37.260ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 50.987195ms iteration = 10 status = {"errors":[],"requests_created_ok":[]}
56512025-03-04T03:22:37.262ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 58.210084ms iteration = 10 status = 0
56522025-03-04T03:22:37.262ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56532025-03-04T03:22:37.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56542025-03-04T03:22:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
56552025-03-04T03:22:37.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56562025-03-04T03:22:37.265ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
56572025-03-04T03:22:37.265ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56582025-03-04T03:22:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56592025-03-04T03:22:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56602025-03-04T03:22:37.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 57.294362ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
56612025-03-04T03:22:37.268ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
56622025-03-04T03:22:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
56632025-03-04T03:22:37.270ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
56642025-03-04T03:22:37.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
56652025-03-04T03:22:37.271ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
56662025-03-04T03:22:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
56672025-03-04T03:22:37.272ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56682025-03-04T03:22:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56692025-03-04T03:22:37.275ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
56702025-03-04T03:22:37.275ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56712025-03-04T03:22:37.276ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2615e48a-7421-40a0-9bde-54326025f0a2", "content-length": "26", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
56722025-03-04T03:22:37.276ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56732025-03-04T03:22:37.277ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56742025-03-04T03:22:37.282ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
56752025-03-04T03:22:37.282ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 78.564193ms iteration = 28 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
56762025-03-04T03:22:37.282ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
56772025-03-04T03:22:37.282ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56782025-03-04T03:22:37.283ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "754fa4b6-a409-42e2-9215-08ad4e58b4a5", "content-length": "26", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
56792025-03-04T03:22:37.283ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56802025-03-04T03:22:37.285ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56812025-03-04T03:22:37.294ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56822025-03-04T03:22:37.294ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
56832025-03-04T03:22:37.294ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 89.333054ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
56842025-03-04T03:22:37.303ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 94.538909ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56852025-03-04T03:22:37.306ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 103.012943ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56862025-03-04T03:22:37.310ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
56872025-03-04T03:22:37.310ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 105.607495ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
56882025-03-04T03:22:37.311ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56892025-03-04T03:22:37.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
56902025-03-04T03:22:37.313ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56912025-03-04T03:22:37.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56922025-03-04T03:22:37.314ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 110.330672ms iteration = 12 status = {"physical_disks_added":0}
56932025-03-04T03:22:37.316ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 106.86354ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
56942025-03-04T03:22:37.320ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
56952025-03-04T03:22:37.320ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56962025-03-04T03:22:37.320ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c777e3d4-a196-4e22-a06f-e7c314741ef0", "content-length": "26", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
56972025-03-04T03:22:37.320ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56982025-03-04T03:22:37.322ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
56992025-03-04T03:22:37.345ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57002025-03-04T03:22:37.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57012025-03-04T03:22:37.350ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
57022025-03-04T03:22:37.350ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 146.331913ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
57032025-03-04T03:22:37.353ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
57042025-03-04T03:22:37.353ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec5e01c2-58e8-47b4-8d14-6443f8eb740f", "content-length": "1", "date": "Tue, 04 Mar 2025 03:22:36 GMT"} })
57052025-03-04T03:22:37.355ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
57062025-03-04T03:22:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
57072025-03-04T03:22:37.373ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 170.972946ms iteration = 10 status = {}
57082025-03-04T03:22:37.396ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
57092025-03-04T03:22:37.396ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57102025-03-04T03:22:37.396ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0eed021-9696-4974-87f4-ac7838ac644c", "content-length": "372", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
57112025-03-04T03:22:37.396ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57122025-03-04T03:22:37.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57132025-03-04T03:22:37.398ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 195.606867ms iteration = 10 status = null
57142025-03-04T03:22:37.411ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57152025-03-04T03:22:37.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57162025-03-04T03:22:37.413ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57172025-03-04T03:22:37.415ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
57182025-03-04T03:22:37.415ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57192025-03-04T03:22:37.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57202025-03-04T03:22:37.418ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57212025-03-04T03:22:37.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
57222025-03-04T03:22:37.419ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57232025-03-04T03:22:37.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57242025-03-04T03:22:37.446ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
57252025-03-04T03:22:37.446ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
57262025-03-04T03:22:37.447ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fa34e9ce-a57b-4403-a76b-4b64a76e58cd", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
57272025-03-04T03:22:37.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57282025-03-04T03:22:37.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57292025-03-04T03:22:37.450ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57302025-03-04T03:22:37.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57312025-03-04T03:22:37.452ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57322025-03-04T03:22:37.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57332025-03-04T03:22:37.456ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
57342025-03-04T03:22:37.456ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57352025-03-04T03:22:37.456ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "73c22914-2c35-472e-90a3-07d0c96507fc", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
57362025-03-04T03:22:37.458ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 253.555417ms iteration = 10 status = {}
57372025-03-04T03:22:37.465ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
57382025-03-04T03:22:37.465ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f4eb025-e0d5-4844-9286-b1c0068a48c7", "content-length": "691", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
57392025-03-04T03:22:37.465ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
57402025-03-04T03:22:37.465ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
57412025-03-04T03:22:37.465ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
57422025-03-04T03:22:37.467ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 264.060386ms iteration = 10 status = {}
57432025-03-04T03:22:37.468ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57442025-03-04T03:22:37.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57452025-03-04T03:22:37.470ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
57462025-03-04T03:22:37.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57472025-03-04T03:22:37.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57482025-03-04T03:22:37.479ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
57492025-03-04T03:22:37.480ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d31a449d-ee86-4bd5-9e97-488636fd64b8", "content-length": "134", "date": "Tue, 04 Mar 2025 03:22:36 GMT"} })
57502025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
57512025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
57522025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
57532025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
57542025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57552025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57562025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57572025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
57582025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57592025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
57602025-03-04T03:22:37.480ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
57612025-03-04T03:22:37.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57622025-03-04T03:22:37.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57632025-03-04T03:22:37.483ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
57642025-03-04T03:22:37.483ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "70336733-bbeb-4a1d-a5c9-59a83804ac19", "content-length": "2", "date": "Tue, 04 Mar 2025 03:22:36 GMT"} })
57652025-03-04T03:22:37.483ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
57662025-03-04T03:22:37.483ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "411a5211-6ddb-4972-a802-5e9ce991c199", "content-length": "51", "date": "Tue, 04 Mar 2025 03:22:36 GMT"} })
57672025-03-04T03:22:37.483ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
57682025-03-04T03:22:37.483ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
57692025-03-04T03:22:37.580ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "763e5ab2-9dcc-4212-be80-bee26047c09e", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
57702025-03-04T03:22:37.580ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
57712025-03-04T03:22:37.580ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
57722025-03-04T03:22:37.583ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dc061be-ca29-4a19-a873-1fb78b0e754a", "content-length": "682", "date": "Tue, 04 Mar 2025 03:22:37 GMT"} })
57732025-03-04T03:22:37.583ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57742025-03-04T03:22:37.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57752025-03-04T03:22:37.587ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57762025-03-04T03:22:37.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57772025-03-04T03:22:37.590ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57782025-03-04T03:22:37.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57792025-03-04T03:22:37.592ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57802025-03-04T03:22:37.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57812025-03-04T03:22:37.599ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 395.078311ms iteration = 10 status = {}
57822025-03-04T03:22:37.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 555 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = d2594898-524b-4653-b369-1df7f28f2cc6 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
57832025-03-04T03:22:37.998ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d45e045-5532-4c62-81a5-095ce79aed4f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57842025-03-04T03:22:37.999ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d45e045-5532-4c62-81a5-095ce79aed4f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57852025-03-04T03:22:38.000ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1830 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d45e045-5532-4c62-81a5-095ce79aed4f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57862025-03-04T03:22:39.019ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12d8ef73-ee16-49bc-a35d-a5a8ec9dce71 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57872025-03-04T03:22:39.020ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12d8ef73-ee16-49bc-a35d-a5a8ec9dce71 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57882025-03-04T03:22:39.021ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2238 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12d8ef73-ee16-49bc-a35d-a5a8ec9dce71 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57892025-03-04T03:22:40.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab766c56-a5b5-4874-976a-dc3fc864eb7b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57902025-03-04T03:22:40.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab766c56-a5b5-4874-976a-dc3fc864eb7b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57912025-03-04T03:22:40.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2096 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab766c56-a5b5-4874-976a-dc3fc864eb7b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57922025-03-04T03:22:41.061ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61b92d54-1539-42ad-bfa9-43018750c12d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57932025-03-04T03:22:41.061ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61b92d54-1539-42ad-bfa9-43018750c12d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57942025-03-04T03:22:41.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2227 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61b92d54-1539-42ad-bfa9-43018750c12d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57952025-03-04T03:22:42.082ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3520572e-0893-4cb8-9613-70d726d95c22 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57962025-03-04T03:22:42.082ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3520572e-0893-4cb8-9613-70d726d95c22 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57972025-03-04T03:22:42.085ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2587 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3520572e-0893-4cb8-9613-70d726d95c22 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57982025-03-04T03:22:43.104ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e836a04f-612e-481f-b20e-846673543c2d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
57992025-03-04T03:22:43.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e836a04f-612e-481f-b20e-846673543c2d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58002025-03-04T03:22:43.107ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2299 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e836a04f-612e-481f-b20e-846673543c2d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58012025-03-04T03:22:44.126ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a8f3175c-51e7-44f8-a020-8ed04b156ec0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58022025-03-04T03:22:44.126ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a8f3175c-51e7-44f8-a020-8ed04b156ec0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58032025-03-04T03:22:44.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2071 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a8f3175c-51e7-44f8-a020-8ed04b156ec0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58042025-03-04T03:22:45.148ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0ded6be9-8f93-4d61-a38a-5e7996f06321 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58052025-03-04T03:22:45.149ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0ded6be9-8f93-4d61-a38a-5e7996f06321 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58062025-03-04T03:22:45.150ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1567 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0ded6be9-8f93-4d61-a38a-5e7996f06321 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58072025-03-04T03:22:46.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b58bf9b-7f13-46e9-bc25-9586e3928155 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58082025-03-04T03:22:46.169ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b58bf9b-7f13-46e9-bc25-9586e3928155 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58092025-03-04T03:22:46.171ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2229 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b58bf9b-7f13-46e9-bc25-9586e3928155 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58102025-03-04T03:22:47.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e428806-ab95-468a-b998-ec25b7f0caf6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58112025-03-04T03:22:47.191ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e428806-ab95-468a-b998-ec25b7f0caf6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58122025-03-04T03:22:47.193ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2604 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e428806-ab95-468a-b998-ec25b7f0caf6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58132025-03-04T03:22:47.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
58142025-03-04T03:22:47.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58152025-03-04T03:22:47.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58162025-03-04T03:22:47.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
58172025-03-04T03:22:47.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58182025-03-04T03:22:47.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58192025-03-04T03:22:47.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58202025-03-04T03:22:47.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58212025-03-04T03:22:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58222025-03-04T03:22:47.207ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.420793ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58232025-03-04T03:22:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58242025-03-04T03:22:47.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58252025-03-04T03:22:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58262025-03-04T03:22:47.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58272025-03-04T03:22:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58282025-03-04T03:22:47.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58292025-03-04T03:22:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
58302025-03-04T03:22:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58312025-03-04T03:22:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58322025-03-04T03:22:47.230ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
58332025-03-04T03:22:47.230ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.366823ms iteration = 29 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
58342025-03-04T03:22:47.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 587 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = c5117a0d-520b-4918-835a-666469561047 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
58352025-03-04T03:22:48.211ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c179a18e-f9f9-4718-9029-433b54e1e34c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58362025-03-04T03:22:48.211ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c179a18e-f9f9-4718-9029-433b54e1e34c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58372025-03-04T03:22:48.213ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2395 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c179a18e-f9f9-4718-9029-433b54e1e34c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58382025-03-04T03:22:49.232ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3c68753-fa9a-4bd2-891a-c84198aad3f9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58392025-03-04T03:22:49.232ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3c68753-fa9a-4bd2-891a-c84198aad3f9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58402025-03-04T03:22:49.234ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1696 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3c68753-fa9a-4bd2-891a-c84198aad3f9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58412025-03-04T03:22:50.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 404f8189-41bd-4945-92f5-673a43cfaf7e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58422025-03-04T03:22:50.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 404f8189-41bd-4945-92f5-673a43cfaf7e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58432025-03-04T03:22:50.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2303 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 404f8189-41bd-4945-92f5-673a43cfaf7e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58442025-03-04T03:22:51.276ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eeaf5b2f-d010-4e45-aa87-85301be2b207 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58452025-03-04T03:22:51.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eeaf5b2f-d010-4e45-aa87-85301be2b207 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58462025-03-04T03:22:51.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2097 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eeaf5b2f-d010-4e45-aa87-85301be2b207 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58472025-03-04T03:22:52.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ada34162-76c4-41c7-b024-d27e1db6c168 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58482025-03-04T03:22:52.299ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ada34162-76c4-41c7-b024-d27e1db6c168 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58492025-03-04T03:22:52.301ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2618 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ada34162-76c4-41c7-b024-d27e1db6c168 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58502025-03-04T03:22:53.320ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5c68e06-bb20-460e-8cd4-ea8c2c7694ec roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58512025-03-04T03:22:53.321ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5c68e06-bb20-460e-8cd4-ea8c2c7694ec resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58522025-03-04T03:22:53.322ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2138 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5c68e06-bb20-460e-8cd4-ea8c2c7694ec response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58532025-03-04T03:22:54.342ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e717d644-84a1-47eb-84c2-dd6436ec78dc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58542025-03-04T03:22:54.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e717d644-84a1-47eb-84c2-dd6436ec78dc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58552025-03-04T03:22:54.344ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1991 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e717d644-84a1-47eb-84c2-dd6436ec78dc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58562025-03-04T03:22:55.363ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c4fb734-4380-4f56-a10f-e928c6748d40 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58572025-03-04T03:22:55.364ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c4fb734-4380-4f56-a10f-e928c6748d40 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58582025-03-04T03:22:55.365ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1941 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c4fb734-4380-4f56-a10f-e928c6748d40 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58592025-03-04T03:22:56.385ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7e529d0d-1ddc-44fe-91ef-fa467b5a09c1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58602025-03-04T03:22:56.385ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7e529d0d-1ddc-44fe-91ef-fa467b5a09c1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58612025-03-04T03:22:56.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2398 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7e529d0d-1ddc-44fe-91ef-fa467b5a09c1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58622025-03-04T03:22:57.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
58632025-03-04T03:22:57.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58642025-03-04T03:22:57.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58652025-03-04T03:22:57.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
58662025-03-04T03:22:57.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58672025-03-04T03:22:57.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58682025-03-04T03:22:57.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58692025-03-04T03:22:57.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58702025-03-04T03:22:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58712025-03-04T03:22:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58722025-03-04T03:22:57.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58732025-03-04T03:22:57.208ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.431612ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58742025-03-04T03:22:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58752025-03-04T03:22:57.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58762025-03-04T03:22:57.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58772025-03-04T03:22:57.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58782025-03-04T03:22:57.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
58792025-03-04T03:22:57.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58802025-03-04T03:22:57.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58812025-03-04T03:22:57.222ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
58822025-03-04T03:22:57.222ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.964832ms iteration = 30 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
58832025-03-04T03:22:57.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cc926a9-5481-44aa-b9ae-a31ac5cc4651 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58842025-03-04T03:22:57.406ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cc926a9-5481-44aa-b9ae-a31ac5cc4651 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58852025-03-04T03:22:57.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1898 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cc926a9-5481-44aa-b9ae-a31ac5cc4651 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58862025-03-04T03:22:57.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 564 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = bcd590f7-00ce-403e-8dc7-3d4ca4229f21 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
58872025-03-04T03:22:58.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5d88c6c9-1cb6-40da-a3d0-c14d4c5b1932 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58882025-03-04T03:22:58.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5d88c6c9-1cb6-40da-a3d0-c14d4c5b1932 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58892025-03-04T03:22:58.430ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2384 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5d88c6c9-1cb6-40da-a3d0-c14d4c5b1932 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58902025-03-04T03:22:59.449ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da5722e5-b14f-4d98-9d24-73254990043d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58912025-03-04T03:22:59.449ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da5722e5-b14f-4d98-9d24-73254990043d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58922025-03-04T03:22:59.451ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2299 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da5722e5-b14f-4d98-9d24-73254990043d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58932025-03-04T03:23:00.473ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd6c8a60-c6b8-4145-97c9-2f62ab238251 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58942025-03-04T03:23:00.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd6c8a60-c6b8-4145-97c9-2f62ab238251 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58952025-03-04T03:23:00.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2098 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd6c8a60-c6b8-4145-97c9-2f62ab238251 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58962025-03-04T03:23:01.494ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1003014-c45e-4b18-be00-e5c7cfc8daa8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58972025-03-04T03:23:01.494ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1003014-c45e-4b18-be00-e5c7cfc8daa8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58982025-03-04T03:23:01.496ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1997 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c1003014-c45e-4b18-be00-e5c7cfc8daa8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
58992025-03-04T03:23:02.515ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22fbdde6-844b-4a71-b8c7-c449df788fd0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59002025-03-04T03:23:02.515ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22fbdde6-844b-4a71-b8c7-c449df788fd0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59012025-03-04T03:23:02.518ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2556 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22fbdde6-844b-4a71-b8c7-c449df788fd0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59022025-03-04T03:23:03.537ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d8281ed4-4ad3-4e72-abda-939a70e8b0ca roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59032025-03-04T03:23:03.537ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d8281ed4-4ad3-4e72-abda-939a70e8b0ca resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59042025-03-04T03:23:03.539ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d8281ed4-4ad3-4e72-abda-939a70e8b0ca response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59052025-03-04T03:23:04.558ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a5b34242-1e1d-4084-a71f-9f416fb37cae roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59062025-03-04T03:23:04.558ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a5b34242-1e1d-4084-a71f-9f416fb37cae resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59072025-03-04T03:23:04.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1966 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a5b34242-1e1d-4084-a71f-9f416fb37cae response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59082025-03-04T03:23:05.579ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ad1d3020-e544-4734-8842-84d120de4b26 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59092025-03-04T03:23:05.579ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ad1d3020-e544-4734-8842-84d120de4b26 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59102025-03-04T03:23:05.581ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2342 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ad1d3020-e544-4734-8842-84d120de4b26 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59112025-03-04T03:23:06.600ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f0afeeb-e0db-4a93-ab4f-b6898c3d5a8b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59122025-03-04T03:23:06.601ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f0afeeb-e0db-4a93-ab4f-b6898c3d5a8b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59132025-03-04T03:23:06.603ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2487 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f0afeeb-e0db-4a93-ab4f-b6898c3d5a8b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
59142025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
59152025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
59162025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
59172025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59182025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
59192025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
59202025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
59212025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
59222025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
59232025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
59242025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
59252025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
59262025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59272025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
59282025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
59292025-03-04T03:23:07.203ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:13:07.203372351 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
59302025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
59312025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
59322025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
59332025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
59342025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59352025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
59362025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 4 reason = Timeout
59372025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59382025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
59392025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
59402025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
59412025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
59422025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Timeout
59432025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
59442025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59452025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59462025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59472025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
59482025-03-04T03:23:07.203ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
59492025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.94\u{b5}s iteration = 7 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
59502025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
59512025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
59522025-03-04T03:23:07.203ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
59532025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59542025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
59552025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
59562025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59572025-03-04T03:23:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
59582025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
59592025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
59602025-03-04T03:23:07.204ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1299, tv_nsec: 401009625 } })
59612025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
59622025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59632025-03-04T03:23:07.204ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1299, tv_nsec: 401073016 } })
59642025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59652025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
59662025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
59672025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59682025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59692025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
59702025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59712025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59722025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59732025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
59742025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.250145ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
59752025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
59762025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
59772025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59782025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
59792025-03-04T03:23:07.204ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
59802025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
59812025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59822025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59832025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
59842025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59852025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
59862025-03-04T03:23:07.204ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
59872025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59882025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
59892025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
59902025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
59912025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.182975ms iteration = 6 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
59922025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59932025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59942025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59952025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
59962025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59972025-03-04T03:23:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59982025-03-04T03:23:07.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
59992025-03-04T03:23:07.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60002025-03-04T03:23:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
60012025-03-04T03:23:07.206ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
60022025-03-04T03:23:07.206ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
60032025-03-04T03:23:07.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1299, tv_nsec: 403677828 } })
60042025-03-04T03:23:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.098429ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
60052025-03-04T03:23:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
60062025-03-04T03:23:07.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
60072025-03-04T03:23:07.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1299, tv_nsec: 404176374 } })
60082025-03-04T03:23:07.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
60092025-03-04T03:23:07.207ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
60102025-03-04T03:23:07.207ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
60112025-03-04T03:23:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
60122025-03-04T03:23:07.208ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:13:07.203372351 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
60132025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.781279ms iteration = 6 status = {"expiration":"2025-03-04T03:13:07.203372351Z","pruned":[]}
60142025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 2 reason = Timeout
60152025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
60162025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
60172025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
60182025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
60192025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
60202025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
60212025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
60222025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
60232025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
60242025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
60252025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
60262025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 11 reason = Timeout
60272025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
60282025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
60292025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
60302025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
60312025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
60322025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
60332025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
60342025-03-04T03:23:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
60352025-03-04T03:23:07.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60362025-03-04T03:23:07.210ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60372025-03-04T03:23:07.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60382025-03-04T03:23:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60392025-03-04T03:23:07.210ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60402025-03-04T03:23:07.211ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60412025-03-04T03:23:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60422025-03-04T03:23:07.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60432025-03-04T03:23:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60442025-03-04T03:23:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
60452025-03-04T03:23:07.213ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
60462025-03-04T03:23:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
60472025-03-04T03:23:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
60482025-03-04T03:23:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60492025-03-04T03:23:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60502025-03-04T03:23:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
60512025-03-04T03:23:07.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60522025-03-04T03:23:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60532025-03-04T03:23:07.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60542025-03-04T03:23:07.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60552025-03-04T03:23:07.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60562025-03-04T03:23:07.216ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60572025-03-04T03:23:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60582025-03-04T03:23:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
60592025-03-04T03:23:07.217ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60602025-03-04T03:23:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60612025-03-04T03:23:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
60622025-03-04T03:23:07.217ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
60632025-03-04T03:23:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
60642025-03-04T03:23:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60652025-03-04T03:23:07.218ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60662025-03-04T03:23:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60672025-03-04T03:23:07.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60682025-03-04T03:23:07.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
60692025-03-04T03:23:07.221ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60702025-03-04T03:23:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60712025-03-04T03:23:07.221ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
60722025-03-04T03:23:07.221ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
60732025-03-04T03:23:07.221ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
60742025-03-04T03:23:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
60752025-03-04T03:23:07.222ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60762025-03-04T03:23:07.222ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60772025-03-04T03:23:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60782025-03-04T03:23:07.224ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
60792025-03-04T03:23:07.224ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
60802025-03-04T03:23:07.224ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 17.389903ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
60812025-03-04T03:23:07.224ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
60822025-03-04T03:23:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
60832025-03-04T03:23:07.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
60842025-03-04T03:23:07.227ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
60852025-03-04T03:23:07.227ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
60862025-03-04T03:23:07.227ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
60872025-03-04T03:23:07.227ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
60882025-03-04T03:23:07.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
60892025-03-04T03:23:07.227ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.078184ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
60902025-03-04T03:23:07.228ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 20.054806ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
60912025-03-04T03:23:07.229ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60922025-03-04T03:23:07.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60932025-03-04T03:23:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60942025-03-04T03:23:07.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60952025-03-04T03:23:07.230ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 22.405584ms iteration = 11 status = {"errors":[],"requests_created_ok":[]}
60962025-03-04T03:23:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60972025-03-04T03:23:07.231ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60982025-03-04T03:23:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60992025-03-04T03:23:07.232ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 27.878892ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
61002025-03-04T03:23:07.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
61012025-03-04T03:23:07.236ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1299, tv_nsec: 433567444 } })
61022025-03-04T03:23:07.236ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
61032025-03-04T03:23:07.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61042025-03-04T03:23:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61052025-03-04T03:23:07.239ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
61062025-03-04T03:23:07.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
61072025-03-04T03:23:07.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61082025-03-04T03:23:07.240ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61092025-03-04T03:23:07.240ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61102025-03-04T03:23:07.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
61112025-03-04T03:23:07.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61122025-03-04T03:23:07.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
61132025-03-04T03:23:07.245ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61142025-03-04T03:23:07.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61152025-03-04T03:23:07.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61162025-03-04T03:23:07.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
61172025-03-04T03:23:07.246ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61182025-03-04T03:23:07.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
61192025-03-04T03:23:07.249ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61202025-03-04T03:23:07.250ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
61212025-03-04T03:23:07.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
61222025-03-04T03:23:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
61232025-03-04T03:23:07.252ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61242025-03-04T03:23:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61252025-03-04T03:23:07.257ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61262025-03-04T03:23:07.259ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
61272025-03-04T03:23:07.259ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
61282025-03-04T03:23:07.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
61292025-03-04T03:23:07.266ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
61302025-03-04T03:23:07.267ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61312025-03-04T03:23:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
61322025-03-04T03:23:07.269ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
61332025-03-04T03:23:07.269ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
61342025-03-04T03:23:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
61352025-03-04T03:23:07.270ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
61362025-03-04T03:23:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
61372025-03-04T03:23:07.276ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
61382025-03-04T03:23:07.276ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
61392025-03-04T03:23:07.276ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 72.546799ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
61402025-03-04T03:23:07.276ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
61412025-03-04T03:23:07.276ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
61422025-03-04T03:23:07.276ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
61432025-03-04T03:23:07.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61442025-03-04T03:23:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61452025-03-04T03:23:07.278ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
61462025-03-04T03:23:07.278ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 74.659775ms iteration = 31 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
61472025-03-04T03:23:07.279ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
61482025-03-04T03:23:07.279ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 74.541024ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
61492025-03-04T03:23:07.279ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61502025-03-04T03:23:07.280ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
61512025-03-04T03:23:07.280ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
61522025-03-04T03:23:07.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
61532025-03-04T03:23:07.280ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
61542025-03-04T03:23:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
61552025-03-04T03:23:07.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61562025-03-04T03:23:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
61572025-03-04T03:23:07.281ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
61582025-03-04T03:23:07.281ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
61592025-03-04T03:23:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
61602025-03-04T03:23:07.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.584492ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
61612025-03-04T03:23:07.282ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
61622025-03-04T03:23:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
61632025-03-04T03:23:07.282ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
61642025-03-04T03:23:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
61652025-03-04T03:23:07.284ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
61662025-03-04T03:23:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
61672025-03-04T03:23:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
61682025-03-04T03:23:07.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61692025-03-04T03:23:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61702025-03-04T03:23:07.286ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61712025-03-04T03:23:07.287ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
61722025-03-04T03:23:07.287ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
61732025-03-04T03:23:07.287ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 83.20911ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
61742025-03-04T03:23:07.288ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61752025-03-04T03:23:07.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
61762025-03-04T03:23:07.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
61772025-03-04T03:23:07.291ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
61782025-03-04T03:23:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
61792025-03-04T03:23:07.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
61802025-03-04T03:23:07.291ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61812025-03-04T03:23:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61822025-03-04T03:23:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
61832025-03-04T03:23:07.292ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
61842025-03-04T03:23:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
61852025-03-04T03:23:07.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
61862025-03-04T03:23:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
61872025-03-04T03:23:07.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
61882025-03-04T03:23:07.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
61892025-03-04T03:23:07.298ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
61902025-03-04T03:23:07.298ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
61912025-03-04T03:23:07.298ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61922025-03-04T03:23:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61932025-03-04T03:23:07.308ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61942025-03-04T03:23:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
61952025-03-04T03:23:07.312ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
61962025-03-04T03:23:07.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
61972025-03-04T03:23:07.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 109.676664ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
61982025-03-04T03:23:07.335ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61992025-03-04T03:23:07.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
62002025-03-04T03:23:07.365ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
62012025-03-04T03:23:07.365ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
62022025-03-04T03:23:07.365ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bab4db72-de55-4bde-b812-561f59e66e1b", "content-length": "26", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
62032025-03-04T03:23:07.365ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
62042025-03-04T03:23:07.367ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
62052025-03-04T03:23:07.371ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 167.534943ms iteration = 13 status = {"physical_disks_added":0}
62062025-03-04T03:23:07.453ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
62072025-03-04T03:23:07.504ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
62082025-03-04T03:23:07.521ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
62092025-03-04T03:23:07.521ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
62102025-03-04T03:23:07.521ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6acf3031-d395-46af-bc05-9de81541ea6b", "content-length": "26", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
62112025-03-04T03:23:07.521ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
62122025-03-04T03:23:07.523ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
62132025-03-04T03:23:07.556ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
62142025-03-04T03:23:07.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
62152025-03-04T03:23:07.559ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62162025-03-04T03:23:07.560ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62172025-03-04T03:23:07.562ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
62182025-03-04T03:23:07.562ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
62192025-03-04T03:23:07.562ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bddd314f-64b1-482f-9baa-9cfb31eb9c53", "content-length": "26", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
62202025-03-04T03:23:07.562ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
62212025-03-04T03:23:07.563ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
62222025-03-04T03:23:07.563ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
62232025-03-04T03:23:07.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
62242025-03-04T03:23:07.564ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
62252025-03-04T03:23:07.564ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62262025-03-04T03:23:07.564ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
62272025-03-04T03:23:07.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
62282025-03-04T03:23:07.571ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62292025-03-04T03:23:07.573ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 369.844843ms iteration = 11 status = 0
62302025-03-04T03:23:07.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
62312025-03-04T03:23:07.574ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
62322025-03-04T03:23:07.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
62332025-03-04T03:23:07.582ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
62342025-03-04T03:23:07.588ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "a519e4ab-1dc2-4cde-a271-9da60d35cee4", "content-length": "2", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
62352025-03-04T03:23:07.590ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:615 sled = http://[fd00:1122:3344:101::1]:12345
62362025-03-04T03:23:07.594ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 385.746407ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
62372025-03-04T03:23:07.595ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 386.710699ms iteration = 2 status = {"last_run_counters":{"copy_err":0,"copy_ok":0,"delete_err":0,"delete_ok":0,"list_err":0,"list_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"delete_err":0,"delete_ok":0,"list_err":0,"list_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"operation":"list","target_sled":"5525e426-7299-43ab-ad2b-346968b24e1c","time":"2025-03-04T03:23:07.590439585Z"}]}
62382025-03-04T03:23:07.596ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
62392025-03-04T03:23:07.596ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
62402025-03-04T03:23:07.596ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62412025-03-04T03:23:07.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62422025-03-04T03:23:07.597ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 389.403063ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
62432025-03-04T03:23:07.598ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62442025-03-04T03:23:07.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62452025-03-04T03:23:07.601ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62462025-03-04T03:23:07.602ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62472025-03-04T03:23:07.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62482025-03-04T03:23:07.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62492025-03-04T03:23:07.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
62502025-03-04T03:23:07.603ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62512025-03-04T03:23:07.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62522025-03-04T03:23:07.603ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62532025-03-04T03:23:07.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62542025-03-04T03:23:07.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62552025-03-04T03:23:07.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
62562025-03-04T03:23:07.605ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62572025-03-04T03:23:07.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62582025-03-04T03:23:07.605ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62592025-03-04T03:23:07.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62602025-03-04T03:23:07.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62612025-03-04T03:23:07.606ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
62622025-03-04T03:23:07.606ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
62632025-03-04T03:23:07.606ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
62642025-03-04T03:23:07.606ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 402.938309ms iteration = 6 status = {"generation":1}
62652025-03-04T03:23:07.607ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
62662025-03-04T03:23:07.607ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
62672025-03-04T03:23:07.607ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
62682025-03-04T03:23:07.607ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 404.297875ms iteration = 6 status = {"generation":2}
62692025-03-04T03:23:07.613ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
62702025-03-04T03:23:07.613ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 409.713191ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
62712025-03-04T03:23:07.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62722025-03-04T03:23:07.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62732025-03-04T03:23:07.615ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62742025-03-04T03:23:07.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
62752025-03-04T03:23:07.617ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62762025-03-04T03:23:07.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
62772025-03-04T03:23:07.621ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3e3bb07f-adf1-4dbd-bf2e-d0af58d94b13 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
62782025-03-04T03:23:07.622ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3e3bb07f-adf1-4dbd-bf2e-d0af58d94b13 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
62792025-03-04T03:23:07.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2200 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3e3bb07f-adf1-4dbd-bf2e-d0af58d94b13 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
62802025-03-04T03:23:07.628ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
62812025-03-04T03:23:07.628ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "c378ef25-6cbb-40cf-8996-3b21f8bc2336", "content-length": "1", "date": "Tue, 04 Mar 2025 03:23:06 GMT"} })
62822025-03-04T03:23:07.629ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
62832025-03-04T03:23:07.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
62842025-03-04T03:23:07.631ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 427.992966ms iteration = 11 status = null
62852025-03-04T03:23:07.639ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62862025-03-04T03:23:07.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62872025-03-04T03:23:07.641ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62882025-03-04T03:23:07.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
62892025-03-04T03:23:07.646ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62902025-03-04T03:23:07.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62912025-03-04T03:23:07.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62922025-03-04T03:23:07.648ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62932025-03-04T03:23:07.649ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62942025-03-04T03:23:07.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62952025-03-04T03:23:07.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62962025-03-04T03:23:07.651ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62972025-03-04T03:23:07.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62982025-03-04T03:23:07.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62992025-03-04T03:23:07.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63002025-03-04T03:23:07.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
63012025-03-04T03:23:07.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
63022025-03-04T03:23:07.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
63032025-03-04T03:23:07.659ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
63042025-03-04T03:23:07.659ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 456.716447ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
63052025-03-04T03:23:07.661ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63062025-03-04T03:23:07.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63072025-03-04T03:23:07.666ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
63082025-03-04T03:23:07.666ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 461.651758ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
63092025-03-04T03:23:07.675ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
63102025-03-04T03:23:07.675ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7dc5c988-d308-4b26-a3de-cda30f2d41af", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
63112025-03-04T03:23:07.675ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
63122025-03-04T03:23:07.677ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ff0b6c03-cfe2-486a-b1f5-7d89363fd0a1", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
63132025-03-04T03:23:07.677ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
63142025-03-04T03:23:07.678ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63152025-03-04T03:23:07.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63162025-03-04T03:23:07.678ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
63172025-03-04T03:23:07.678ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 475.029951ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
63182025-03-04T03:23:07.690ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 487.09581ms iteration = 11 status = {}
63192025-03-04T03:23:07.718ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
63202025-03-04T03:23:07.738ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
63212025-03-04T03:23:07.738ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
63222025-03-04T03:23:07.739ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a2289fe-c243-43fa-b23e-59c4e15754b4", "content-length": "372", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
63232025-03-04T03:23:07.739ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63242025-03-04T03:23:07.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63252025-03-04T03:23:07.741ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63262025-03-04T03:23:07.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63272025-03-04T03:23:07.741ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
63282025-03-04T03:23:07.742ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 538.47627ms iteration = 4 status = {"blueprint_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","inventory_collection_id":"ebea1c1f-7f7f-45d0-b2be-3da6745abd56","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
63292025-03-04T03:23:07.743ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63302025-03-04T03:23:07.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63312025-03-04T03:23:07.745ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63322025-03-04T03:23:07.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63332025-03-04T03:23:07.780ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63342025-03-04T03:23:07.782ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
63352025-03-04T03:23:07.782ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63362025-03-04T03:23:07.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63372025-03-04T03:23:07.783ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63382025-03-04T03:23:07.784ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63392025-03-04T03:23:07.785ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
63402025-03-04T03:23:07.785ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63412025-03-04T03:23:07.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63422025-03-04T03:23:07.787ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
63432025-03-04T03:23:07.787ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63442025-03-04T03:23:07.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63452025-03-04T03:23:07.789ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63462025-03-04T03:23:07.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63472025-03-04T03:23:07.791ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
63482025-03-04T03:23:07.791ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
63492025-03-04T03:23:07.791ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63502025-03-04T03:23:07.792ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "11f22683-41ca-488d-b623-391fc96088a5", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
63512025-03-04T03:23:07.792ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63522025-03-04T03:23:07.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63532025-03-04T03:23:07.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63542025-03-04T03:23:07.794ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63552025-03-04T03:23:07.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63562025-03-04T03:23:07.796ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63572025-03-04T03:23:07.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63582025-03-04T03:23:07.797ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63592025-03-04T03:23:07.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63602025-03-04T03:23:07.798ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63612025-03-04T03:23:07.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63622025-03-04T03:23:07.800ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63632025-03-04T03:23:07.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63642025-03-04T03:23:07.800ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
63652025-03-04T03:23:07.800ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
63662025-03-04T03:23:07.801ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "feb170cb-59c6-449e-991c-ded9dcbb7ded", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
63672025-03-04T03:23:07.801ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63682025-03-04T03:23:07.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63692025-03-04T03:23:07.803ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 599.033762ms iteration = 11 status = {}
63702025-03-04T03:23:07.803ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63712025-03-04T03:23:07.807ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
63722025-03-04T03:23:07.807ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63732025-03-04T03:23:07.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63742025-03-04T03:23:07.808ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63752025-03-04T03:23:07.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63762025-03-04T03:23:07.810ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63772025-03-04T03:23:07.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63782025-03-04T03:23:07.811ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63792025-03-04T03:23:07.815ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
63802025-03-04T03:23:07.815ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63812025-03-04T03:23:07.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63822025-03-04T03:23:07.825ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63832025-03-04T03:23:07.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63842025-03-04T03:23:07.826ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63852025-03-04T03:23:07.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63862025-03-04T03:23:07.828ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63872025-03-04T03:23:07.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63882025-03-04T03:23:07.830ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63892025-03-04T03:23:07.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63902025-03-04T03:23:07.833ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63912025-03-04T03:23:07.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63922025-03-04T03:23:07.835ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63932025-03-04T03:23:07.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63942025-03-04T03:23:07.836ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63952025-03-04T03:23:07.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63962025-03-04T03:23:07.838ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63972025-03-04T03:23:07.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
63982025-03-04T03:23:07.841ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63992025-03-04T03:23:07.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
64002025-03-04T03:23:07.843ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
64012025-03-04T03:23:07.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
64022025-03-04T03:23:07.844ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
64032025-03-04T03:23:07.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
64042025-03-04T03:23:07.846ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64052025-03-04T03:23:07.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
64062025-03-04T03:23:07.849ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
64072025-03-04T03:23:07.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
64082025-03-04T03:23:07.863ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
64092025-03-04T03:23:07.864ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "ec50c003-98d6-49ef-9794-728cce083977", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64102025-03-04T03:23:07.864ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64112025-03-04T03:23:07.866ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5784f2b1-3d12-426a-bc03-919981665136", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64122025-03-04T03:23:07.866ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64132025-03-04T03:23:07.868ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "119779a3-b2dc-48d7-b1c9-00280369bb7c", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64142025-03-04T03:23:07.868ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64152025-03-04T03:23:07.869ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64162025-03-04T03:23:07.869ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 666.730822ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
64172025-03-04T03:23:07.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 458 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 3cbb1af3-cdd4-46c9-8973-3704fb06d2a9 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
64182025-03-04T03:23:07.912ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
64192025-03-04T03:23:07.912ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0d13cd6-609d-4f86-9aa4-17b2791c976b", "content-length": "691", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
64202025-03-04T03:23:07.912ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
64212025-03-04T03:23:07.912ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
64222025-03-04T03:23:07.912ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
64232025-03-04T03:23:07.913ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 709.783089ms iteration = 11 status = {}
64242025-03-04T03:23:07.922ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 1efc0800-e2a1-43fb-8159-014fa2b2d648, kind: Service { id: 72700cb3-c938-46f0-9b9c-46cf389d975e }, name: Name("external-dns-72700cb3-c938-46f0-9b9c-46cf389d975e"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 246])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6aab6111-ac14-4d7e-808e-18a2dbaec1a3, kind: Service { id: 0e2dad65-6466-48b5-bc3a-8a9f5178c9b7 }, name: Name("external-dns-0e2dad65-6466-48b5-bc3a-8a9f5178c9b7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 05a86b80-7d75-4c2d-8253-0cdbdce17c69, kind: Service { id: 2afd3cd1-228b-4949-95e6-cb3413cc5ec2 }, name: Name("nexus-2afd3cd1-228b-4949-95e6-cb3413cc5ec2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 61])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3ce82d4d-d64c-418d-9d43-fc9e659efafa, kind: Service { id: c28b34ab-1b39-4325-87ad-c4ccf5b009d5 }, name: Name("nexus-c28b34ab-1b39-4325-87ad-c4ccf5b009d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 509b968a-6934-4b81-be1c-676fe22e2948, kind: Service { id: d8484452-7d43-42ac-a588-575b6d31b882 }, name: Name("nexus-d8484452-7d43-42ac-a588-575b6d31b882"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 93])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
64252025-03-04T03:23:07.922ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
64262025-03-04T03:23:07.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
64272025-03-04T03:23:07.924ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64282025-03-04T03:23:07.926ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
64292025-03-04T03:23:07.926ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
64302025-03-04T03:23:07.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
64312025-03-04T03:23:07.929ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64322025-03-04T03:23:07.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64332025-03-04T03:23:07.931ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
64342025-03-04T03:23:07.931ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64352025-03-04T03:23:07.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64362025-03-04T03:23:07.937ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
64372025-03-04T03:23:07.937ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1efc0800-e2a1-43fb-8159-014fa2b2d648, kind: Service { id: 72700cb3-c938-46f0-9b9c-46cf389d975e }, name: Name("external-dns-72700cb3-c938-46f0-9b9c-46cf389d975e"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 246])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6aab6111-ac14-4d7e-808e-18a2dbaec1a3, kind: Service { id: 0e2dad65-6466-48b5-bc3a-8a9f5178c9b7 }, name: Name("external-dns-0e2dad65-6466-48b5-bc3a-8a9f5178c9b7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 05a86b80-7d75-4c2d-8253-0cdbdce17c69, kind: Service { id: 2afd3cd1-228b-4949-95e6-cb3413cc5ec2 }, name: Name("nexus-2afd3cd1-228b-4949-95e6-cb3413cc5ec2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 61])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3ce82d4d-d64c-418d-9d43-fc9e659efafa, kind: Service { id: c28b34ab-1b39-4325-87ad-c4ccf5b009d5 }, name: Name("nexus-c28b34ab-1b39-4325-87ad-c4ccf5b009d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 509b968a-6934-4b81-be1c-676fe22e2948, kind: Service { id: d8484452-7d43-42ac-a588-575b6d31b882 }, name: Name("nexus-d8484452-7d43-42ac-a588-575b6d31b882"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 93])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
64382025-03-04T03:23:07.937ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
64392025-03-04T03:23:07.941ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
64402025-03-04T03:23:07.941ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "27d9a5a4-c0bf-4bd2-b415-b897f3e78dae", "content-length": "134", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
64412025-03-04T03:23:07.941ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
64422025-03-04T03:23:07.941ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
64432025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
64442025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
64452025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64462025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64472025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64482025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
64492025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64502025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
64512025-03-04T03:23:07.942ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
64522025-03-04T03:23:07.942ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64532025-03-04T03:23:07.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64542025-03-04T03:23:07.944ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
64552025-03-04T03:23:07.945ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0799fd78-cd7c-4b12-870b-e6552bdf1911", "content-length": "2", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
64562025-03-04T03:23:07.945ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
64572025-03-04T03:23:07.945ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "527c599d-0dbe-4e05-87be-d3dcd63b8043", "content-length": "51", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
64582025-03-04T03:23:07.945ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
64592025-03-04T03:23:07.945ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
64602025-03-04T03:23:08.040ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "16cc2114-7351-4082-9128-6e11f5fbd7ed", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
64612025-03-04T03:23:08.040ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
64622025-03-04T03:23:08.040ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
64632025-03-04T03:23:08.041ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "60b43097-f55e-48e1-8b52-f04a2daa6864", "content-length": "682", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
64642025-03-04T03:23:08.041ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64652025-03-04T03:23:08.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64662025-03-04T03:23:08.045ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64672025-03-04T03:23:08.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64682025-03-04T03:23:08.049ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64692025-03-04T03:23:08.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64702025-03-04T03:23:08.051ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64712025-03-04T03:23:08.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64722025-03-04T03:23:08.056ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 5525e426-7299-43ab-ad2b-346968b24e1c, time_created: 2025-03-04T03:18:30.785539Z, time_modified: 2025-03-04T03:18:30.785539Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
64732025-03-04T03:23:08.056ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
64742025-03-04T03:23:08.056ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
64752025-03-04T03:23:08.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
64762025-03-04T03:23:08.058ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 854.908067ms iteration = 11 status = {}
64772025-03-04T03:23:08.058ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
64782025-03-04T03:23:08.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
64792025-03-04T03:23:08.060ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64802025-03-04T03:23:08.062ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
64812025-03-04T03:23:08.097ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
64822025-03-04T03:23:08.098ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "3477ad87-206f-4e04-83c8-30125a45ed9e", "date": "Tue, 04 Mar 2025 03:23:07 GMT"} })
64832025-03-04T03:23:08.098ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
64842025-03-04T03:23:08.098ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 894.391042ms
64852025-03-04T03:23:08.098ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 894.400912ms iteration = 2 status = {}
64862025-03-04T03:23:08.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d58d967d-97b3-49f3-be6c-8e313c282904 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64872025-03-04T03:23:08.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d58d967d-97b3-49f3-be6c-8e313c282904 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64882025-03-04T03:23:08.645ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2728 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d58d967d-97b3-49f3-be6c-8e313c282904 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64892025-03-04T03:23:09.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e13c848c-a371-4203-b908-46b58828fea0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64902025-03-04T03:23:09.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e13c848c-a371-4203-b908-46b58828fea0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64912025-03-04T03:23:09.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2575 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e13c848c-a371-4203-b908-46b58828fea0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64922025-03-04T03:23:10.689ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e1aefa39-f4c0-4835-a720-e6ccc1be5c06 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64932025-03-04T03:23:10.690ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e1aefa39-f4c0-4835-a720-e6ccc1be5c06 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64942025-03-04T03:23:10.692ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2448 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e1aefa39-f4c0-4835-a720-e6ccc1be5c06 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64952025-03-04T03:23:11.712ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d32e8004-4173-45d3-9295-2638bba56368 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64962025-03-04T03:23:11.713ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d32e8004-4173-45d3-9295-2638bba56368 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64972025-03-04T03:23:11.715ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2968 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d32e8004-4173-45d3-9295-2638bba56368 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64982025-03-04T03:23:12.734ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd4a3242-c31b-4dbb-bafe-4910c435c4e6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
64992025-03-04T03:23:12.735ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd4a3242-c31b-4dbb-bafe-4910c435c4e6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65002025-03-04T03:23:12.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3209 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd4a3242-c31b-4dbb-bafe-4910c435c4e6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65012025-03-04T03:23:13.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09d99e99-4863-4686-8cd8-928901d8ae21 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65022025-03-04T03:23:13.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09d99e99-4863-4686-8cd8-928901d8ae21 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65032025-03-04T03:23:13.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2418 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09d99e99-4863-4686-8cd8-928901d8ae21 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65042025-03-04T03:23:14.778ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c05eb97-23ca-4d09-b87a-d0953212994e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65052025-03-04T03:23:14.778ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c05eb97-23ca-4d09-b87a-d0953212994e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65062025-03-04T03:23:14.782ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3678 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c05eb97-23ca-4d09-b87a-d0953212994e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65072025-03-04T03:23:15.801ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ffdec39e-f460-4a74-9c6b-1b2bdac12958 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65082025-03-04T03:23:15.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ffdec39e-f460-4a74-9c6b-1b2bdac12958 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65092025-03-04T03:23:15.803ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2116 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ffdec39e-f460-4a74-9c6b-1b2bdac12958 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65102025-03-04T03:23:16.821ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f583f7f9-126e-4364-a74e-6ac0c74506be roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65112025-03-04T03:23:16.822ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f583f7f9-126e-4364-a74e-6ac0c74506be resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65122025-03-04T03:23:16.823ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2321 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f583f7f9-126e-4364-a74e-6ac0c74506be response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65132025-03-04T03:23:17.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
65142025-03-04T03:23:17.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65152025-03-04T03:23:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65162025-03-04T03:23:17.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
65172025-03-04T03:23:17.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65182025-03-04T03:23:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65192025-03-04T03:23:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65202025-03-04T03:23:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65212025-03-04T03:23:17.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65222025-03-04T03:23:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65232025-03-04T03:23:17.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65242025-03-04T03:23:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65252025-03-04T03:23:17.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65262025-03-04T03:23:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
65272025-03-04T03:23:17.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65282025-03-04T03:23:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65292025-03-04T03:23:17.227ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
65302025-03-04T03:23:17.227ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.407057ms iteration = 32 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
65312025-03-04T03:23:17.234ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65322025-03-04T03:23:17.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65332025-03-04T03:23:17.264ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.315837ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65342025-03-04T03:23:17.841ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 76c8d9fe-5454-4c8c-9a21-4ced1daae422 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65352025-03-04T03:23:17.842ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 76c8d9fe-5454-4c8c-9a21-4ced1daae422 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65362025-03-04T03:23:17.843ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2111 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 76c8d9fe-5454-4c8c-9a21-4ced1daae422 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65372025-03-04T03:23:17.889ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:39552
65382025-03-04T03:23:17.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = a7465a6f-98d5-4ab6-8fbe-736efca315ec roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
65392025-03-04T03:23:17.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = a7465a6f-98d5-4ab6-8fbe-736efca315ec resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
65402025-03-04T03:23:17.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3017 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = a7465a6f-98d5-4ab6-8fbe-736efca315ec response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
65412025-03-04T03:23:17.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 25db9317-78f7-4aee-b9fb-3ae66a170365 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
65422025-03-04T03:23:17.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 25db9317-78f7-4aee-b9fb-3ae66a170365 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
65432025-03-04T03:23:17.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1676 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 25db9317-78f7-4aee-b9fb-3ae66a170365 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
65442025-03-04T03:23:17.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 548 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = a3c9ecce-82f7-4e46-a241-3b371c246313 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
65452025-03-04T03:23:18.862ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3228a86f-0f43-4d9c-9dc3-37c6a58be5b7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65462025-03-04T03:23:18.862ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3228a86f-0f43-4d9c-9dc3-37c6a58be5b7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65472025-03-04T03:23:18.864ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2028 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3228a86f-0f43-4d9c-9dc3-37c6a58be5b7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65482025-03-04T03:23:19.883ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae5dba55-5a8a-41b9-b203-9200f47c2e79 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65492025-03-04T03:23:19.883ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae5dba55-5a8a-41b9-b203-9200f47c2e79 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65502025-03-04T03:23:19.885ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2321 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae5dba55-5a8a-41b9-b203-9200f47c2e79 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65512025-03-04T03:23:20.904ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d751d212-3d0c-4a6d-8927-fb397b8ab4c0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65522025-03-04T03:23:20.905ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d751d212-3d0c-4a6d-8927-fb397b8ab4c0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65532025-03-04T03:23:20.906ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2217 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d751d212-3d0c-4a6d-8927-fb397b8ab4c0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65542025-03-04T03:23:21.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b10b09ba-a3ba-416c-833f-14f6398518f5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65552025-03-04T03:23:21.925ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b10b09ba-a3ba-416c-833f-14f6398518f5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65562025-03-04T03:23:21.927ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1890 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b10b09ba-a3ba-416c-833f-14f6398518f5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65572025-03-04T03:23:22.947ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d71f7e8-952f-4e1a-9a81-d491e1a0f3df roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65582025-03-04T03:23:22.948ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d71f7e8-952f-4e1a-9a81-d491e1a0f3df resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65592025-03-04T03:23:22.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1857 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d71f7e8-952f-4e1a-9a81-d491e1a0f3df response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65602025-03-04T03:23:23.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7b00c61-fb02-4a43-ad66-7d56e83a1167 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65612025-03-04T03:23:23.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7b00c61-fb02-4a43-ad66-7d56e83a1167 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65622025-03-04T03:23:23.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2356 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7b00c61-fb02-4a43-ad66-7d56e83a1167 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65632025-03-04T03:23:24.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 833c22c9-ba72-47c8-b631-e55298371183 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65642025-03-04T03:23:24.992ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 833c22c9-ba72-47c8-b631-e55298371183 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65652025-03-04T03:23:24.994ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2215 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 833c22c9-ba72-47c8-b631-e55298371183 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65662025-03-04T03:23:26.013ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be47669f-ebd8-406f-9c18-32cef6f9edde roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65672025-03-04T03:23:26.014ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be47669f-ebd8-406f-9c18-32cef6f9edde resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65682025-03-04T03:23:26.016ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2446 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be47669f-ebd8-406f-9c18-32cef6f9edde response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65692025-03-04T03:23:27.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 92adc53a-c415-4fa3-9c8e-dbb2470ad7af roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65702025-03-04T03:23:27.036ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 92adc53a-c415-4fa3-9c8e-dbb2470ad7af resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65712025-03-04T03:23:27.038ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2172 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 92adc53a-c415-4fa3-9c8e-dbb2470ad7af response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65722025-03-04T03:23:27.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
65732025-03-04T03:23:27.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65742025-03-04T03:23:27.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65752025-03-04T03:23:27.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
65762025-03-04T03:23:27.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65772025-03-04T03:23:27.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65782025-03-04T03:23:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65792025-03-04T03:23:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65802025-03-04T03:23:27.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65812025-03-04T03:23:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65822025-03-04T03:23:27.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65832025-03-04T03:23:27.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65842025-03-04T03:23:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65852025-03-04T03:23:27.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
65862025-03-04T03:23:27.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65872025-03-04T03:23:27.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65882025-03-04T03:23:27.232ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
65892025-03-04T03:23:27.232ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.088364ms iteration = 33 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
65902025-03-04T03:23:27.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65912025-03-04T03:23:27.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65922025-03-04T03:23:27.267ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.170234ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65932025-03-04T03:23:27.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 646 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = a2ca7c6a-3f2b-40e1-b75e-1d739983ebca response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
65942025-03-04T03:23:28.059ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ba1a8fd0-facf-45ed-8a6a-23a4643f9cab roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65952025-03-04T03:23:28.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ba1a8fd0-facf-45ed-8a6a-23a4643f9cab resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65962025-03-04T03:23:28.061ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2293 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ba1a8fd0-facf-45ed-8a6a-23a4643f9cab response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65972025-03-04T03:23:29.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cfc1ae64-67a2-4d5b-beb7-c89c831bb808 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65982025-03-04T03:23:29.081ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cfc1ae64-67a2-4d5b-beb7-c89c831bb808 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
65992025-03-04T03:23:29.082ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2151 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cfc1ae64-67a2-4d5b-beb7-c89c831bb808 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66002025-03-04T03:23:30.101ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c845718-59d4-4e9b-8b12-16adfba9dc9a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66012025-03-04T03:23:30.101ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c845718-59d4-4e9b-8b12-16adfba9dc9a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66022025-03-04T03:23:30.103ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2398 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c845718-59d4-4e9b-8b12-16adfba9dc9a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66032025-03-04T03:23:31.122ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eed17c4d-11dd-4705-af17-1630b0da213d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66042025-03-04T03:23:31.123ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eed17c4d-11dd-4705-af17-1630b0da213d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66052025-03-04T03:23:31.124ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2049 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eed17c4d-11dd-4705-af17-1630b0da213d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66062025-03-04T03:23:32.143ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d815ddca-3d5c-434b-9e6a-1e94c7703554 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66072025-03-04T03:23:32.143ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d815ddca-3d5c-434b-9e6a-1e94c7703554 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66082025-03-04T03:23:32.145ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2172 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d815ddca-3d5c-434b-9e6a-1e94c7703554 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66092025-03-04T03:23:33.164ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc23e40e-0a88-46c8-8095-e7781acf6330 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66102025-03-04T03:23:33.165ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc23e40e-0a88-46c8-8095-e7781acf6330 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66112025-03-04T03:23:33.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1886 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc23e40e-0a88-46c8-8095-e7781acf6330 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66122025-03-04T03:23:34.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5e2512c6-84c4-4167-b92f-2295cf9eebae roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66132025-03-04T03:23:34.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5e2512c6-84c4-4167-b92f-2295cf9eebae resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66142025-03-04T03:23:34.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2726 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5e2512c6-84c4-4167-b92f-2295cf9eebae response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66152025-03-04T03:23:35.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e079a119-a8f4-4687-9664-56c68efa41e8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66162025-03-04T03:23:35.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e079a119-a8f4-4687-9664-56c68efa41e8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66172025-03-04T03:23:35.210ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2243 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e079a119-a8f4-4687-9664-56c68efa41e8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66182025-03-04T03:23:36.229ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 842e96ca-f5b7-449c-aaad-59b397405ba4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66192025-03-04T03:23:36.229ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 842e96ca-f5b7-449c-aaad-59b397405ba4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66202025-03-04T03:23:36.231ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2505 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 842e96ca-f5b7-449c-aaad-59b397405ba4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
66212025-03-04T03:23:37.202ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
66222025-03-04T03:23:37.202ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
66232025-03-04T03:23:37.202ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
66242025-03-04T03:23:37.202ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
66252025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
66262025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
66272025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
66282025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
66292025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
66302025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66312025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
66322025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66332025-03-04T03:23:37.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66342025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
66352025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66362025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
66372025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
66382025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
66392025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
66402025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
66412025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66422025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66432025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66442025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66452025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66462025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66472025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
66482025-03-04T03:23:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
66492025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
66502025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
66512025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
66522025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66532025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
66542025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
66552025-03-04T03:23:37.205ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
66562025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66572025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
66582025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
66592025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
66602025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66612025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66622025-03-04T03:23:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
66632025-03-04T03:23:37.205ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1329, tv_nsec: 402716488 } })
66642025-03-04T03:23:37.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1329, tv_nsec: 402771529 } })
66652025-03-04T03:23:37.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
66662025-03-04T03:23:37.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66672025-03-04T03:23:37.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
66682025-03-04T03:23:37.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66692025-03-04T03:23:37.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66702025-03-04T03:23:37.207ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.917098ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
66712025-03-04T03:23:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
66722025-03-04T03:23:37.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66732025-03-04T03:23:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
66742025-03-04T03:23:37.208ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66752025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
66762025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66772025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66782025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66792025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66802025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
66812025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
66822025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
66832025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
66842025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 12 reason = Timeout
66852025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
66862025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
66872025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
66882025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
66892025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
66902025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
66912025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
66922025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
66932025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
66942025-03-04T03:23:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
66952025-03-04T03:23:37.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66962025-03-04T03:23:37.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66972025-03-04T03:23:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66982025-03-04T03:23:37.212ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66992025-03-04T03:23:37.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67002025-03-04T03:23:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67012025-03-04T03:23:37.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67022025-03-04T03:23:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67032025-03-04T03:23:37.214ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67042025-03-04T03:23:37.214ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67052025-03-04T03:23:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67062025-03-04T03:23:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
67072025-03-04T03:23:37.214ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
67082025-03-04T03:23:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
67092025-03-04T03:23:37.215ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
67102025-03-04T03:23:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
67112025-03-04T03:23:37.217ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1329, tv_nsec: 413889375 } })
67122025-03-04T03:23:37.217ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
67132025-03-04T03:23:37.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67142025-03-04T03:23:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
67152025-03-04T03:23:37.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67162025-03-04T03:23:37.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67172025-03-04T03:23:37.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67182025-03-04T03:23:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
67192025-03-04T03:23:37.246ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67202025-03-04T03:23:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67212025-03-04T03:23:37.248ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
67222025-03-04T03:23:37.248ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 44.476195ms iteration = 34 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
67232025-03-04T03:23:37.249ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67242025-03-04T03:23:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67252025-03-04T03:23:37.251ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67262025-03-04T03:23:37.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f295ec95-7050-4ab2-8dd1-e9d322201578 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
67272025-03-04T03:23:37.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f295ec95-7050-4ab2-8dd1-e9d322201578 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
67282025-03-04T03:23:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
67292025-03-04T03:23:37.253ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67302025-03-04T03:23:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67312025-03-04T03:23:37.254ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
67322025-03-04T03:23:37.254ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
67332025-03-04T03:23:37.254ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
67342025-03-04T03:23:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
67352025-03-04T03:23:37.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3391 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f295ec95-7050-4ab2-8dd1-e9d322201578 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
67362025-03-04T03:23:37.258ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67372025-03-04T03:23:37.258ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 49.317915ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
67382025-03-04T03:23:37.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67392025-03-04T03:23:37.260ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67402025-03-04T03:23:37.261ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 51.610272ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
67412025-03-04T03:23:37.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67422025-03-04T03:23:37.261ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
67432025-03-04T03:23:37.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
67442025-03-04T03:23:37.262ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67452025-03-04T03:23:37.263ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67462025-03-04T03:23:37.263ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
67472025-03-04T03:23:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
67482025-03-04T03:23:37.265ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 56.205689ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
67492025-03-04T03:23:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
67502025-03-04T03:23:37.265ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67512025-03-04T03:23:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
67522025-03-04T03:23:37.265ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67532025-03-04T03:23:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67542025-03-04T03:23:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67552025-03-04T03:23:37.267ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67562025-03-04T03:23:37.268ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.78335ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67572025-03-04T03:23:37.268ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 64.314928ms iteration = 12 status = 0
67582025-03-04T03:23:37.268ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
67592025-03-04T03:23:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
67602025-03-04T03:23:37.269ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67612025-03-04T03:23:37.269ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67622025-03-04T03:23:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67632025-03-04T03:23:37.271ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
67642025-03-04T03:23:37.271ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 66.272783ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
67652025-03-04T03:23:37.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 68.411679ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
67662025-03-04T03:23:37.277ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 68.51921ms iteration = 12 status = {"errors":[],"requests_created_ok":[]}
67672025-03-04T03:23:37.290ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
67682025-03-04T03:23:37.290ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
67692025-03-04T03:23:37.290ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
67702025-03-04T03:23:37.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
67712025-03-04T03:23:37.297ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67722025-03-04T03:23:37.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67732025-03-04T03:23:37.299ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 90.167485ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
67742025-03-04T03:23:37.312ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
67752025-03-04T03:23:37.312ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
67762025-03-04T03:23:37.312ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a1ed312-b31f-4c8d-ac32-340c2efe2063", "content-length": "26", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
67772025-03-04T03:23:37.312ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
67782025-03-04T03:23:37.314ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
67792025-03-04T03:23:37.331ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67802025-03-04T03:23:37.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67812025-03-04T03:23:37.357ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 153.826285ms iteration = 14 status = {"physical_disks_added":0}
67822025-03-04T03:23:37.389ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
67832025-03-04T03:23:37.389ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
67842025-03-04T03:23:37.390ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bca2a0a4-22c8-4621-945d-7d7ddc9df15f", "content-length": "372", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
67852025-03-04T03:23:37.390ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67862025-03-04T03:23:37.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67872025-03-04T03:23:37.401ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
67882025-03-04T03:23:37.401ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
67892025-03-04T03:23:37.401ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a924cf3-6823-4ca9-b60f-145b52ca89bb", "content-length": "26", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
67902025-03-04T03:23:37.401ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
67912025-03-04T03:23:37.404ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
67922025-03-04T03:23:37.411ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
67932025-03-04T03:23:37.412ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c9bfe8f-ddeb-4981-ac3f-2c3effd5bf88", "content-length": "1", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
67942025-03-04T03:23:37.413ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
67952025-03-04T03:23:37.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
67962025-03-04T03:23:37.414ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
67972025-03-04T03:23:37.414ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
67982025-03-04T03:23:37.415ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "63ecfc25-f7f2-4a8d-bcfe-9b1cff08d1c3", "content-length": "26", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
67992025-03-04T03:23:37.415ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
68002025-03-04T03:23:37.415ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 212.95463ms iteration = 12 status = null
68012025-03-04T03:23:37.416ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
68022025-03-04T03:23:37.430ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
68032025-03-04T03:23:37.430ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
68042025-03-04T03:23:37.430ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 224.704224ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
68052025-03-04T03:23:37.438ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68062025-03-04T03:23:37.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
68072025-03-04T03:23:37.440ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
68082025-03-04T03:23:37.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
68092025-03-04T03:23:37.449ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
68102025-03-04T03:23:37.449ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
68112025-03-04T03:23:37.450ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b82494a0-90ea-4e4c-b59b-e3c830b9e87b", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
68122025-03-04T03:23:37.450ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68132025-03-04T03:23:37.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68142025-03-04T03:23:37.460ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68152025-03-04T03:23:37.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68162025-03-04T03:23:37.464ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68172025-03-04T03:23:37.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68182025-03-04T03:23:37.468ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
68192025-03-04T03:23:37.468ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
68202025-03-04T03:23:37.468ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b67b2725-73b8-4513-ada2-a68fec0ae6e5", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
68212025-03-04T03:23:37.470ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 265.365562ms iteration = 12 status = {}
68222025-03-04T03:23:37.474ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 271.693979ms iteration = 12 status = {}
68232025-03-04T03:23:37.477ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
68242025-03-04T03:23:37.477ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 273.698883ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
68252025-03-04T03:23:37.498ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68262025-03-04T03:23:37.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68272025-03-04T03:23:37.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68282025-03-04T03:23:37.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
68292025-03-04T03:23:37.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68302025-03-04T03:23:37.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68312025-03-04T03:23:37.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68322025-03-04T03:23:37.533ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
68332025-03-04T03:23:37.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68342025-03-04T03:23:37.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68352025-03-04T03:23:37.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68362025-03-04T03:23:37.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68372025-03-04T03:23:37.536ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
68382025-03-04T03:23:37.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68392025-03-04T03:23:37.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68402025-03-04T03:23:37.546ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
68412025-03-04T03:23:37.546ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e903424-835b-4633-b87c-5438e36cb4a5", "content-length": "134", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
68422025-03-04T03:23:37.546ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
68432025-03-04T03:23:37.546ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
68442025-03-04T03:23:37.546ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
68452025-03-04T03:23:37.546ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
68462025-03-04T03:23:37.547ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68472025-03-04T03:23:37.547ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68482025-03-04T03:23:37.547ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68492025-03-04T03:23:37.547ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
68502025-03-04T03:23:37.547ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68512025-03-04T03:23:37.547ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
68522025-03-04T03:23:37.547ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
68532025-03-04T03:23:37.547ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68542025-03-04T03:23:37.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68552025-03-04T03:23:37.549ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
68562025-03-04T03:23:37.549ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad3a0704-6d61-4970-8035-4aa7ec585736", "content-length": "2", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
68572025-03-04T03:23:37.549ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
68582025-03-04T03:23:37.549ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae1a2635-edeb-4efa-ade6-94343d7c6c83", "content-length": "51", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
68592025-03-04T03:23:37.549ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
68602025-03-04T03:23:37.549ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
68612025-03-04T03:23:37.624ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
68622025-03-04T03:23:37.644ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "be68caf3-c4a2-4f29-bcf5-969b11281696", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
68632025-03-04T03:23:37.644ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
68642025-03-04T03:23:37.644ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
68652025-03-04T03:23:37.644ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "636c149f-8f9d-47bc-b6c9-93bfef4e5b86", "content-length": "691", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
68662025-03-04T03:23:37.644ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
68672025-03-04T03:23:37.644ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
68682025-03-04T03:23:37.644ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
68692025-03-04T03:23:37.644ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a8f7bb7-f142-443a-b668-a3fff535bb24", "content-length": "682", "date": "Tue, 04 Mar 2025 03:23:37 GMT"} })
68702025-03-04T03:23:37.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68712025-03-04T03:23:37.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68722025-03-04T03:23:37.645ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 441.175316ms iteration = 12 status = {}
68732025-03-04T03:23:37.648ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68742025-03-04T03:23:37.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68752025-03-04T03:23:37.652ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68762025-03-04T03:23:37.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68772025-03-04T03:23:37.654ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68782025-03-04T03:23:37.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68792025-03-04T03:23:37.661ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 457.688048ms iteration = 12 status = {}
68802025-03-04T03:23:37.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 475 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 5a074ca5-8575-4786-bb8a-aedec4e5c235 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
68812025-03-04T03:23:38.276ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 519931b7-e610-424f-8a5e-539e8d979140 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68822025-03-04T03:23:38.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 519931b7-e610-424f-8a5e-539e8d979140 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68832025-03-04T03:23:38.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2201 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 519931b7-e610-424f-8a5e-539e8d979140 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68842025-03-04T03:23:39.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1a73692-2b31-42b6-928d-3362db79a03c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68852025-03-04T03:23:39.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1a73692-2b31-42b6-928d-3362db79a03c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68862025-03-04T03:23:39.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2506 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1a73692-2b31-42b6-928d-3362db79a03c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68872025-03-04T03:23:40.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 86bab95c-c597-4c38-ab7a-ab52e78f9fbf roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68882025-03-04T03:23:40.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 86bab95c-c597-4c38-ab7a-ab52e78f9fbf resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68892025-03-04T03:23:40.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2624 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 86bab95c-c597-4c38-ab7a-ab52e78f9fbf response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68902025-03-04T03:23:41.338ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e1ae557-6477-403a-aace-c257f91a35d3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68912025-03-04T03:23:41.338ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e1ae557-6477-403a-aace-c257f91a35d3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68922025-03-04T03:23:41.341ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2765 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e1ae557-6477-403a-aace-c257f91a35d3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68932025-03-04T03:23:42.360ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8ceaa0f3-4a4d-4bce-a402-86612ea48bde roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68942025-03-04T03:23:42.361ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8ceaa0f3-4a4d-4bce-a402-86612ea48bde resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68952025-03-04T03:23:42.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2598 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8ceaa0f3-4a4d-4bce-a402-86612ea48bde response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68962025-03-04T03:23:43.384ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8705d661-14c7-42cb-85ae-c4c624e9a99f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68972025-03-04T03:23:43.384ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8705d661-14c7-42cb-85ae-c4c624e9a99f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68982025-03-04T03:23:43.386ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1859 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8705d661-14c7-42cb-85ae-c4c624e9a99f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
68992025-03-04T03:23:44.404ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c15ce401-f84d-4824-96ea-9f6916468e77 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69002025-03-04T03:23:44.405ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c15ce401-f84d-4824-96ea-9f6916468e77 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69012025-03-04T03:23:44.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2388 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c15ce401-f84d-4824-96ea-9f6916468e77 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69022025-03-04T03:23:45.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8f4b0070-817c-43dd-96d8-54406a52eaf7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69032025-03-04T03:23:45.427ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8f4b0070-817c-43dd-96d8-54406a52eaf7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69042025-03-04T03:23:45.429ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2554 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8f4b0070-817c-43dd-96d8-54406a52eaf7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69052025-03-04T03:23:46.448ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bab7e127-203d-4783-bb68-de3c6cfb2e6a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69062025-03-04T03:23:46.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bab7e127-203d-4783-bb68-de3c6cfb2e6a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69072025-03-04T03:23:46.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2516 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bab7e127-203d-4783-bb68-de3c6cfb2e6a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69082025-03-04T03:23:47.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
69092025-03-04T03:23:47.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69102025-03-04T03:23:47.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69112025-03-04T03:23:47.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
69122025-03-04T03:23:47.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69132025-03-04T03:23:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69142025-03-04T03:23:47.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69152025-03-04T03:23:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
69162025-03-04T03:23:47.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69172025-03-04T03:23:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69182025-03-04T03:23:47.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69192025-03-04T03:23:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69202025-03-04T03:23:47.234ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69212025-03-04T03:23:47.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69222025-03-04T03:23:47.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69232025-03-04T03:23:47.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
69242025-03-04T03:23:47.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69252025-03-04T03:23:47.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69262025-03-04T03:23:47.257ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
69272025-03-04T03:23:47.257ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 53.06515ms iteration = 35 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
69282025-03-04T03:23:47.270ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.674455ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
69292025-03-04T03:23:47.469ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 873f353f-1d93-473a-ad3e-2eb50f764c78 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69302025-03-04T03:23:47.470ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 873f353f-1d93-473a-ad3e-2eb50f764c78 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69312025-03-04T03:23:47.471ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2417 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 873f353f-1d93-473a-ad3e-2eb50f764c78 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69322025-03-04T03:23:47.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 5f1232db-d6e2-4542-bbbe-89bddb76d0a1 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
69332025-03-04T03:23:47.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 5f1232db-d6e2-4542-bbbe-89bddb76d0a1 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
69342025-03-04T03:23:47.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3567 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 5f1232db-d6e2-4542-bbbe-89bddb76d0a1 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
69352025-03-04T03:23:47.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 47ac89df-4a76-46a7-9f35-7ef90786febe roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
69362025-03-04T03:23:47.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 47ac89df-4a76-46a7-9f35-7ef90786febe resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
69372025-03-04T03:23:47.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2302 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 47ac89df-4a76-46a7-9f35-7ef90786febe response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
69382025-03-04T03:23:47.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 444 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = a957e01d-ac1f-4fb5-bb02-c48a7bc9221d response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
69392025-03-04T03:23:48.490ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 469cd78a-f5c7-47af-87b0-92fe46c1ebad roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69402025-03-04T03:23:48.491ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 469cd78a-f5c7-47af-87b0-92fe46c1ebad resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69412025-03-04T03:23:48.492ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1886 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 469cd78a-f5c7-47af-87b0-92fe46c1ebad response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69422025-03-04T03:23:49.512ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 98f01bd1-c767-4a45-9dea-2387b1a978eb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69432025-03-04T03:23:49.512ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 98f01bd1-c767-4a45-9dea-2387b1a978eb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69442025-03-04T03:23:49.513ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1765 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 98f01bd1-c767-4a45-9dea-2387b1a978eb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69452025-03-04T03:23:50.534ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48155b3d-db0d-4b49-9073-96d492b68065 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69462025-03-04T03:23:50.534ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48155b3d-db0d-4b49-9073-96d492b68065 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69472025-03-04T03:23:50.536ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2494 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48155b3d-db0d-4b49-9073-96d492b68065 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69482025-03-04T03:23:51.555ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84458187-112a-4d54-8876-a73ed5e15b65 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69492025-03-04T03:23:51.556ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84458187-112a-4d54-8876-a73ed5e15b65 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69502025-03-04T03:23:51.558ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2519 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84458187-112a-4d54-8876-a73ed5e15b65 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69512025-03-04T03:23:52.578ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc7673fd-48e6-48d7-8f75-b9bfbc8300d6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69522025-03-04T03:23:52.578ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc7673fd-48e6-48d7-8f75-b9bfbc8300d6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69532025-03-04T03:23:52.581ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2673 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc7673fd-48e6-48d7-8f75-b9bfbc8300d6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69542025-03-04T03:23:53.600ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 37d1c50a-78fc-4981-af56-353572c579a8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69552025-03-04T03:23:53.601ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 37d1c50a-78fc-4981-af56-353572c579a8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69562025-03-04T03:23:53.603ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2259 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 37d1c50a-78fc-4981-af56-353572c579a8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69572025-03-04T03:23:54.622ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f34391e-794f-4e14-925b-a0dd153fc68c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69582025-03-04T03:23:54.622ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f34391e-794f-4e14-925b-a0dd153fc68c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69592025-03-04T03:23:54.624ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2119 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0f34391e-794f-4e14-925b-a0dd153fc68c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69602025-03-04T03:23:55.643ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 54bdfae3-728c-442a-995e-a58e16f43755 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69612025-03-04T03:23:55.644ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 54bdfae3-728c-442a-995e-a58e16f43755 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69622025-03-04T03:23:55.645ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2044 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 54bdfae3-728c-442a-995e-a58e16f43755 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69632025-03-04T03:23:56.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84bcdb1c-6353-43c3-a821-ce173f1a6707 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69642025-03-04T03:23:56.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84bcdb1c-6353-43c3-a821-ce173f1a6707 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69652025-03-04T03:23:56.667ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2352 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84bcdb1c-6353-43c3-a821-ce173f1a6707 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69662025-03-04T03:23:57.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
69672025-03-04T03:23:57.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69682025-03-04T03:23:57.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69692025-03-04T03:23:57.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
69702025-03-04T03:23:57.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69712025-03-04T03:23:57.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69722025-03-04T03:23:57.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69732025-03-04T03:23:57.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69742025-03-04T03:23:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69752025-03-04T03:23:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
69762025-03-04T03:23:57.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69772025-03-04T03:23:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69782025-03-04T03:23:57.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69792025-03-04T03:23:57.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69802025-03-04T03:23:57.237ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.849413ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
69812025-03-04T03:23:57.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69822025-03-04T03:23:57.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
69832025-03-04T03:23:57.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69842025-03-04T03:23:57.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69852025-03-04T03:23:57.253ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
69862025-03-04T03:23:57.253ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.309914ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
69872025-03-04T03:23:57.686ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b10db611-8679-4abf-8ec3-cdc3b0f874df roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69882025-03-04T03:23:57.686ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b10db611-8679-4abf-8ec3-cdc3b0f874df resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69892025-03-04T03:23:57.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1825 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b10db611-8679-4abf-8ec3-cdc3b0f874df response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69902025-03-04T03:23:57.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 465 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = b3574ccd-caba-4063-b8c5-eeb35d3d68d7 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
69912025-03-04T03:23:58.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 116aea99-8a82-4fab-9e99-93b23c9142f4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69922025-03-04T03:23:58.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 116aea99-8a82-4fab-9e99-93b23c9142f4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69932025-03-04T03:23:58.708ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1677 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 116aea99-8a82-4fab-9e99-93b23c9142f4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69942025-03-04T03:23:59.727ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42fbb683-76c1-4056-a7fa-35b4ce301e2d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69952025-03-04T03:23:59.728ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42fbb683-76c1-4056-a7fa-35b4ce301e2d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69962025-03-04T03:23:59.730ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2657 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42fbb683-76c1-4056-a7fa-35b4ce301e2d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69972025-03-04T03:24:00.749ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b820dcae-6547-4324-a793-ba2091024730 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69982025-03-04T03:24:00.750ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b820dcae-6547-4324-a793-ba2091024730 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
69992025-03-04T03:24:00.751ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2019 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b820dcae-6547-4324-a793-ba2091024730 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70002025-03-04T03:24:01.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88171731-3654-478a-900a-137f9cedaa27 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70012025-03-04T03:24:01.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88171731-3654-478a-900a-137f9cedaa27 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70022025-03-04T03:24:01.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1825 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88171731-3654-478a-900a-137f9cedaa27 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70032025-03-04T03:24:02.792ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec129afb-0035-4a7a-933a-14c883b67f12 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70042025-03-04T03:24:02.792ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec129afb-0035-4a7a-933a-14c883b67f12 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70052025-03-04T03:24:02.794ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2435 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec129afb-0035-4a7a-933a-14c883b67f12 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70062025-03-04T03:24:02.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = b16f5525-6989-431a-a38c-3f5ce31dde68 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
70072025-03-04T03:24:02.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = b16f5525-6989-431a-a38c-3f5ce31dde68 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
70082025-03-04T03:24:02.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3642 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = b16f5525-6989-431a-a38c-3f5ce31dde68 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
70092025-03-04T03:24:02.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 9a52c7e0-186d-45d3-8f83-2e1264521f6f roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
70102025-03-04T03:24:02.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 9a52c7e0-186d-45d3-8f83-2e1264521f6f resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
70112025-03-04T03:24:02.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1951 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 9a52c7e0-186d-45d3-8f83-2e1264521f6f response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
70122025-03-04T03:24:03.813ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 683682a3-edd4-48ea-957f-caed889177cd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70132025-03-04T03:24:03.814ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 683682a3-edd4-48ea-957f-caed889177cd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70142025-03-04T03:24:03.816ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2615 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 683682a3-edd4-48ea-957f-caed889177cd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70152025-03-04T03:24:04.835ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 252fc524-4d99-4395-b964-da4199c6641a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70162025-03-04T03:24:04.835ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 252fc524-4d99-4395-b964-da4199c6641a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70172025-03-04T03:24:04.837ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2275 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 252fc524-4d99-4395-b964-da4199c6641a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70182025-03-04T03:24:05.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3b6f3e4d-0220-44e1-8180-f3c75fce48f9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70192025-03-04T03:24:05.857ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3b6f3e4d-0220-44e1-8180-f3c75fce48f9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70202025-03-04T03:24:05.859ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2037 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3b6f3e4d-0220-44e1-8180-f3c75fce48f9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70212025-03-04T03:24:06.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e614a8b4-45fb-4d1a-931d-8333794842d8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70222025-03-04T03:24:06.878ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e614a8b4-45fb-4d1a-931d-8333794842d8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70232025-03-04T03:24:06.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1854 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e614a8b4-45fb-4d1a-931d-8333794842d8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
70242025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
70252025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
70262025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
70272025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
70282025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
70292025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70302025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
70312025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
70322025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70332025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
70342025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
70352025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
70362025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
70372025-03-04T03:24:07.203ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:14:07.203381101 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
70382025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
70392025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
70402025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
70412025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
70422025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
70432025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70442025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
70452025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
70462025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
70472025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
70482025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
70492025-03-04T03:24:07.203ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1359, tv_nsec: 400454612 } })
70502025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
70512025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
70522025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
70532025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
70542025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70552025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
70562025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
70572025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70582025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
70592025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70602025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
70612025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
70622025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
70632025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
70642025-03-04T03:24:07.203ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
70652025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70662025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.45\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
70672025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
70682025-03-04T03:24:07.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70692025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
70702025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
70712025-03-04T03:24:07.204ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
70722025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
70732025-03-04T03:24:07.204ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1359, tv_nsec: 400646465 } })
70742025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
70752025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
70762025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70772025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70782025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70792025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
70802025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70812025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70822025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70832025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70842025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
70852025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
70862025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
70872025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.141063ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
70882025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.104533ms iteration = 7 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
70892025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70902025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70912025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70922025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70932025-03-04T03:24:07.204ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70942025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70952025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70962025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70972025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
70982025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
70992025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
71002025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
71012025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71022025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
71032025-03-04T03:24:07.205ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
71042025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
71052025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
71062025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
71072025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
71082025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
71092025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
71102025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
71112025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
71122025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71132025-03-04T03:24:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
71142025-03-04T03:24:07.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71152025-03-04T03:24:07.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71162025-03-04T03:24:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71172025-03-04T03:24:07.206ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
71182025-03-04T03:24:07.206ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
71192025-03-04T03:24:07.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1359, tv_nsec: 403674982 } })
71202025-03-04T03:24:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
71212025-03-04T03:24:07.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
71222025-03-04T03:24:07.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1359, tv_nsec: 404154738 } })
71232025-03-04T03:24:07.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
71242025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
71252025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71262025-03-04T03:24:07.208ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:14:07.203381101 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
71272025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.094072ms iteration = 7 status = {"expiration":"2025-03-04T03:14:07.203381101Z","pruned":[]}
71282025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71292025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
71302025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
71312025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
71322025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
71332025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
71342025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
71352025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
71362025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
71372025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
71382025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
71392025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
71402025-03-04T03:24:07.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
71412025-03-04T03:24:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
71422025-03-04T03:24:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
71432025-03-04T03:24:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
71442025-03-04T03:24:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
71452025-03-04T03:24:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
71462025-03-04T03:24:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
71472025-03-04T03:24:07.211ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.33863ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
71482025-03-04T03:24:07.211ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71492025-03-04T03:24:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71502025-03-04T03:24:07.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71512025-03-04T03:24:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71522025-03-04T03:24:07.215ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71532025-03-04T03:24:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71542025-03-04T03:24:07.217ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71552025-03-04T03:24:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71562025-03-04T03:24:07.229ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
71572025-03-04T03:24:07.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
71582025-03-04T03:24:07.247ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71592025-03-04T03:24:07.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71602025-03-04T03:24:07.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
71612025-03-04T03:24:07.249ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71622025-03-04T03:24:07.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71632025-03-04T03:24:07.250ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71642025-03-04T03:24:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
71652025-03-04T03:24:07.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71662025-03-04T03:24:07.252ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71672025-03-04T03:24:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71682025-03-04T03:24:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71692025-03-04T03:24:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
71702025-03-04T03:24:07.254ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
71712025-03-04T03:24:07.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
71722025-03-04T03:24:07.257ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71732025-03-04T03:24:07.260ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 51.564132ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
71742025-03-04T03:24:07.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
71752025-03-04T03:24:07.260ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71762025-03-04T03:24:07.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71772025-03-04T03:24:07.261ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71782025-03-04T03:24:07.263ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71792025-03-04T03:24:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71802025-03-04T03:24:07.264ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71812025-03-04T03:24:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71822025-03-04T03:24:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71832025-03-04T03:24:07.264ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71842025-03-04T03:24:07.265ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71852025-03-04T03:24:07.265ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71862025-03-04T03:24:07.266ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71872025-03-04T03:24:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
71882025-03-04T03:24:07.267ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
71892025-03-04T03:24:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71902025-03-04T03:24:07.268ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71912025-03-04T03:24:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
71922025-03-04T03:24:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71932025-03-04T03:24:07.268ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71942025-03-04T03:24:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
71952025-03-04T03:24:07.268ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71962025-03-04T03:24:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71972025-03-04T03:24:07.269ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71982025-03-04T03:24:07.269ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
71992025-03-04T03:24:07.269ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72002025-03-04T03:24:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72012025-03-04T03:24:07.270ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72022025-03-04T03:24:07.270ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
72032025-03-04T03:24:07.270ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72042025-03-04T03:24:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72052025-03-04T03:24:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
72062025-03-04T03:24:07.270ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
72072025-03-04T03:24:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
72082025-03-04T03:24:07.270ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72092025-03-04T03:24:07.271ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72102025-03-04T03:24:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
72112025-03-04T03:24:07.273ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
72122025-03-04T03:24:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
72132025-03-04T03:24:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
72142025-03-04T03:24:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
72152025-03-04T03:24:07.274ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
72162025-03-04T03:24:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
72172025-03-04T03:24:07.276ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72182025-03-04T03:24:07.276ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72192025-03-04T03:24:07.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72202025-03-04T03:24:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72212025-03-04T03:24:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72222025-03-04T03:24:07.278ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72232025-03-04T03:24:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72242025-03-04T03:24:07.279ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
72252025-03-04T03:24:07.279ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
72262025-03-04T03:24:07.279ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 72.188065ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
72272025-03-04T03:24:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
72282025-03-04T03:24:07.280ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72292025-03-04T03:24:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72302025-03-04T03:24:07.284ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1359, tv_nsec: 481193832 } })
72312025-03-04T03:24:07.284ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72322025-03-04T03:24:07.284ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
72332025-03-04T03:24:07.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
72342025-03-04T03:24:07.286ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72352025-03-04T03:24:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72362025-03-04T03:24:07.287ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 83.672905ms iteration = 4 status = {"nsuccess":5}
72372025-03-04T03:24:07.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72382025-03-04T03:24:07.291ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72392025-03-04T03:24:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72402025-03-04T03:24:07.293ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72412025-03-04T03:24:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72422025-03-04T03:24:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
72432025-03-04T03:24:07.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72442025-03-04T03:24:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72452025-03-04T03:24:07.302ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 93.365294ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
72462025-03-04T03:24:07.309ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
72472025-03-04T03:24:07.309ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
72482025-03-04T03:24:07.309ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
72492025-03-04T03:24:07.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72502025-03-04T03:24:07.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72512025-03-04T03:24:07.309ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 100.936527ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
72522025-03-04T03:24:07.310ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
72532025-03-04T03:24:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
72542025-03-04T03:24:07.311ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 102.152332ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
72552025-03-04T03:24:07.312ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
72562025-03-04T03:24:07.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72572025-03-04T03:24:07.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
72582025-03-04T03:24:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
72592025-03-04T03:24:07.315ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
72602025-03-04T03:24:07.315ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
72612025-03-04T03:24:07.315ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
72622025-03-04T03:24:07.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
72632025-03-04T03:24:07.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
72642025-03-04T03:24:07.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
72652025-03-04T03:24:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
72662025-03-04T03:24:07.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 116.154684ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
72672025-03-04T03:24:07.320ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
72682025-03-04T03:24:07.320ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 115.246832ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
72692025-03-04T03:24:07.321ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72702025-03-04T03:24:07.321ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
72712025-03-04T03:24:07.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72722025-03-04T03:24:07.321ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
72732025-03-04T03:24:07.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72742025-03-04T03:24:07.323ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72752025-03-04T03:24:07.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72762025-03-04T03:24:07.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 118.134097ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
72772025-03-04T03:24:07.324ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
72782025-03-04T03:24:07.326ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
72792025-03-04T03:24:07.326ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
72802025-03-04T03:24:07.326ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 122.928117ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
72812025-03-04T03:24:07.329ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72822025-03-04T03:24:07.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72832025-03-04T03:24:07.335ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
72842025-03-04T03:24:07.335ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72852025-03-04T03:24:07.336ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1ba315e-9b92-4952-beb9-8f2091f57e8c", "content-length": "26", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
72862025-03-04T03:24:07.336ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72872025-03-04T03:24:07.338ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
72882025-03-04T03:24:07.339ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
72892025-03-04T03:24:07.339ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72902025-03-04T03:24:07.339ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "84510c64-1a02-484f-a61b-d454fd04e17f", "content-length": "26", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
72912025-03-04T03:24:07.339ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72922025-03-04T03:24:07.340ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
72932025-03-04T03:24:07.383ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72942025-03-04T03:24:07.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
72952025-03-04T03:24:07.493ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
72962025-03-04T03:24:07.493ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "58d80c4a-cc97-4f2c-bb85-801b170e2380", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
72972025-03-04T03:24:07.493ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
72982025-03-04T03:24:07.495ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "12644d12-260a-4c3f-8d7f-9e07b2c1c671", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
72992025-03-04T03:24:07.495ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73002025-03-04T03:24:07.496ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73012025-03-04T03:24:07.496ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 293.239284ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
73022025-03-04T03:24:07.531ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
73032025-03-04T03:24:07.531ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "88c937e3-a1ed-473d-8ad5-60de64598650", "content-length": "1", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
73042025-03-04T03:24:07.533ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
73052025-03-04T03:24:07.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
73062025-03-04T03:24:07.552ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
73072025-03-04T03:24:07.552ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73082025-03-04T03:24:07.552ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4c45cfc-484c-4285-a472-54058f32658d", "content-length": "26", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
73092025-03-04T03:24:07.552ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
73102025-03-04T03:24:07.554ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
73112025-03-04T03:24:07.579ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
73122025-03-04T03:24:07.579ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
73132025-03-04T03:24:07.580ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ab529b4-1f72-44cd-b549-5a02dd8a9ce1", "content-length": "372", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
73142025-03-04T03:24:07.580ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73152025-03-04T03:24:07.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73162025-03-04T03:24:07.582ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73172025-03-04T03:24:07.583ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73182025-03-04T03:24:07.585ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
73192025-03-04T03:24:07.585ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
73202025-03-04T03:24:07.585ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
73212025-03-04T03:24:07.585ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
73222025-03-04T03:24:07.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
73232025-03-04T03:24:07.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
73242025-03-04T03:24:07.586ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73252025-03-04T03:24:07.588ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73262025-03-04T03:24:07.588ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73272025-03-04T03:24:07.589ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73282025-03-04T03:24:07.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73292025-03-04T03:24:07.590ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 386.594148ms iteration = 13 status = null
73302025-03-04T03:24:07.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
73312025-03-04T03:24:07.590ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73322025-03-04T03:24:07.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73332025-03-04T03:24:07.590ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
73342025-03-04T03:24:07.590ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
73352025-03-04T03:24:07.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
73362025-03-04T03:24:07.591ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
73372025-03-04T03:24:07.591ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73382025-03-04T03:24:07.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73392025-03-04T03:24:07.593ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73402025-03-04T03:24:07.594ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
73412025-03-04T03:24:07.594ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 390.067571ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
73422025-03-04T03:24:07.594ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73432025-03-04T03:24:07.596ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
73442025-03-04T03:24:07.596ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73452025-03-04T03:24:07.596ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
73462025-03-04T03:24:07.596ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73472025-03-04T03:24:07.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73482025-03-04T03:24:07.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73492025-03-04T03:24:07.598ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 394.587056ms iteration = 13 status = 0
73502025-03-04T03:24:07.619ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
73512025-03-04T03:24:07.620ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 411.814857ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
73522025-03-04T03:24:07.622ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 418.881504ms iteration = 13 status = {}
73532025-03-04T03:24:07.624ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73542025-03-04T03:24:07.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73552025-03-04T03:24:07.626ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73562025-03-04T03:24:07.626ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
73572025-03-04T03:24:07.626ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 421.018929ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
73582025-03-04T03:24:07.634ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
73592025-03-04T03:24:07.634ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 431.083764ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
73602025-03-04T03:24:07.636ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
73612025-03-04T03:24:07.636ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
73622025-03-04T03:24:07.636ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
73632025-03-04T03:24:07.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
73642025-03-04T03:24:07.637ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
73652025-03-04T03:24:07.637ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
73662025-03-04T03:24:07.637ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73672025-03-04T03:24:07.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73682025-03-04T03:24:07.641ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
73692025-03-04T03:24:07.641ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 438.063349ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
73702025-03-04T03:24:07.642ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73712025-03-04T03:24:07.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
73722025-03-04T03:24:07.645ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73732025-03-04T03:24:07.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73742025-03-04T03:24:07.648ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
73752025-03-04T03:24:07.648ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
73762025-03-04T03:24:07.648ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
73772025-03-04T03:24:07.648ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 444.776301ms iteration = 7 status = {"generation":2}
73782025-03-04T03:24:07.650ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
73792025-03-04T03:24:07.650ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
73802025-03-04T03:24:07.650ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d8e72ed6-cee0-4b37-93d5-bfdc3b2635e2", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
73812025-03-04T03:24:07.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73822025-03-04T03:24:07.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73832025-03-04T03:24:07.653ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73842025-03-04T03:24:07.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73852025-03-04T03:24:07.655ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 446.603614ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
73862025-03-04T03:24:07.656ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73872025-03-04T03:24:07.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73882025-03-04T03:24:07.658ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
73892025-03-04T03:24:07.658ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
73902025-03-04T03:24:07.659ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6b2d7e6e-298f-4a4e-8f19-53dd7766f6da", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
73912025-03-04T03:24:07.661ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 456.055239ms iteration = 13 status = {}
73922025-03-04T03:24:07.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73932025-03-04T03:24:07.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
73942025-03-04T03:24:07.670ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
73952025-03-04T03:24:07.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
73962025-03-04T03:24:07.678ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
73972025-03-04T03:24:07.678ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
73982025-03-04T03:24:07.678ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
73992025-03-04T03:24:07.678ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 474.975361ms iteration = 7 status = {"generation":1}
74002025-03-04T03:24:07.683ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
74012025-03-04T03:24:07.683ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 478.550295ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
74022025-03-04T03:24:07.687ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 483.721259ms iteration = 15 status = {"physical_disks_added":0}
74032025-03-04T03:24:07.695ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
74042025-03-04T03:24:07.696ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7273c904-f438-48d6-ac14-c7e7ff86184b", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
74052025-03-04T03:24:07.696ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
74062025-03-04T03:24:07.697ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9181399b-d783-41fa-a051-bf142cf7c8f9", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
74072025-03-04T03:24:07.697ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
74082025-03-04T03:24:07.699ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "808fa4f6-e0ad-4119-bf9b-b877f3215952", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
74092025-03-04T03:24:07.699ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
74102025-03-04T03:24:07.701ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
74112025-03-04T03:24:07.701ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 498.162925ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
74122025-03-04T03:24:07.735ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
74132025-03-04T03:24:07.736ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a364363-e08d-469a-9b71-aca6c29c8c16", "content-length": "691", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
74142025-03-04T03:24:07.736ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
74152025-03-04T03:24:07.736ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
74162025-03-04T03:24:07.736ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
74172025-03-04T03:24:07.737ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 533.63917ms iteration = 13 status = {}
74182025-03-04T03:24:07.754ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74192025-03-04T03:24:07.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74202025-03-04T03:24:07.756ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
74212025-03-04T03:24:07.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74222025-03-04T03:24:07.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74232025-03-04T03:24:07.766ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
74242025-03-04T03:24:07.766ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3625772-6ae0-4821-a7a5-61ee715cc093", "content-length": "134", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
74252025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
74262025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
74272025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
74282025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
74292025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74302025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74312025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74322025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74332025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74342025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74352025-03-04T03:24:07.766ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
74362025-03-04T03:24:07.766ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74372025-03-04T03:24:07.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74382025-03-04T03:24:07.768ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
74392025-03-04T03:24:07.768ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6e0a0b9-67cb-4249-b96b-680c7065091d", "content-length": "2", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
74402025-03-04T03:24:07.768ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
74412025-03-04T03:24:07.768ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "392647d8-2cd6-4440-b998-56dc780e867b", "content-length": "51", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
74422025-03-04T03:24:07.768ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
74432025-03-04T03:24:07.768ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
74442025-03-04T03:24:07.839ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
74452025-03-04T03:24:07.839ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
74462025-03-04T03:24:07.870ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
74472025-03-04T03:24:07.870ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:33888
74482025-03-04T03:24:07.870ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5979f25f-2eaa-4825-b115-08e39067be92", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
74492025-03-04T03:24:07.870ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
74502025-03-04T03:24:07.870ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
74512025-03-04T03:24:07.870ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:33888 req_id = 67e08e52-4eb0-4d89-99d1-89e14ae809cf roles = RoleSet { roles: {} } uri = /metrics/producers
74522025-03-04T03:24:07.870ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0015aafa-e6b9-49d8-92a9-2bc7e3cc58a9", "content-length": "682", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
74532025-03-04T03:24:07.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74542025-03-04T03:24:07.871ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:33888 req_id = 67e08e52-4eb0-4d89-99d1-89e14ae809cf resource = Database result = Ok(()) uri = /metrics/producers
74552025-03-04T03:24:07.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74562025-03-04T03:24:07.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74572025-03-04T03:24:07.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74582025-03-04T03:24:07.879ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74592025-03-04T03:24:07.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74602025-03-04T03:24:07.881ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74612025-03-04T03:24:07.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74622025-03-04T03:24:07.888ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 684.693641ms iteration = 13 status = {}
74632025-03-04T03:24:07.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 461 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 2665937c-9de0-47c9-9d66-8e0dbe0377db response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
74642025-03-04T03:24:07.901ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8966e20c-3daa-408d-86a1-3a7814d68adf roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74652025-03-04T03:24:07.901ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8966e20c-3daa-408d-86a1-3a7814d68adf resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74662025-03-04T03:24:07.903ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2307 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8966e20c-3daa-408d-86a1-3a7814d68adf response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74672025-03-04T03:24:07.910ZINFOnexus: assigned collector to new producer collector_id = ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a file = nexus/src/app/oximeter.rs:76 producer_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
74682025-03-04T03:24:07.910ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 40241 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:33888 req_id = 67e08e52-4eb0-4d89-99d1-89e14ae809cf response_code = 201 uri = /metrics/producers
74692025-03-04T03:24:07.910ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "67e08e52-4eb0-4d89-99d1-89e14ae809cf", "content-length": "41", "date": "Tue, 04 Mar 2025 03:24:07 GMT"} })
74702025-03-04T03:24:07.913ZDEBGnexus (producer-registration-task): registered with nexus successfully
74712025-03-04T03:24:07.913ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
74722025-03-04T03:24:08.922ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25fca499-b841-4deb-a7cd-b8e7c6067ce2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74732025-03-04T03:24:08.922ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25fca499-b841-4deb-a7cd-b8e7c6067ce2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74742025-03-04T03:24:08.924ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1903 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25fca499-b841-4deb-a7cd-b8e7c6067ce2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74752025-03-04T03:24:09.943ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be4d6979-814b-4428-9af4-ed8191d68e84 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74762025-03-04T03:24:09.944ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be4d6979-814b-4428-9af4-ed8191d68e84 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74772025-03-04T03:24:09.946ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2691 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be4d6979-814b-4428-9af4-ed8191d68e84 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74782025-03-04T03:24:10.966ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8e4486c4-a174-453b-bb19-61ec63aad33c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74792025-03-04T03:24:10.967ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8e4486c4-a174-453b-bb19-61ec63aad33c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74802025-03-04T03:24:10.969ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2834 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8e4486c4-a174-453b-bb19-61ec63aad33c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74812025-03-04T03:24:11.988ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0d22031-191a-4f21-87b2-77d69f592495 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74822025-03-04T03:24:11.988ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0d22031-191a-4f21-87b2-77d69f592495 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74832025-03-04T03:24:11.990ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2621 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b0d22031-191a-4f21-87b2-77d69f592495 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74842025-03-04T03:24:13.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e0a0650-fe6f-4cb0-8601-40721c32f8ea roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74852025-03-04T03:24:13.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e0a0650-fe6f-4cb0-8601-40721c32f8ea resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74862025-03-04T03:24:13.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2413 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4e0a0650-fe6f-4cb0-8601-40721c32f8ea response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74872025-03-04T03:24:14.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e10e8f30-043a-4dd2-a18c-20e5164f92cb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74882025-03-04T03:24:14.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e10e8f30-043a-4dd2-a18c-20e5164f92cb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74892025-03-04T03:24:14.032ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1903 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e10e8f30-043a-4dd2-a18c-20e5164f92cb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74902025-03-04T03:24:15.052ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed5d1f32-56ff-4eb5-b337-292e1dbf6bb8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74912025-03-04T03:24:15.052ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed5d1f32-56ff-4eb5-b337-292e1dbf6bb8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74922025-03-04T03:24:15.054ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2478 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed5d1f32-56ff-4eb5-b337-292e1dbf6bb8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74932025-03-04T03:24:16.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c0dd0e0-f91d-4c21-841c-d66ff009bf53 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74942025-03-04T03:24:16.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c0dd0e0-f91d-4c21-841c-d66ff009bf53 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74952025-03-04T03:24:16.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2190 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c0dd0e0-f91d-4c21-841c-d66ff009bf53 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74962025-03-04T03:24:17.095ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5e97d35-aa16-4d99-abc7-c898768b0a11 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74972025-03-04T03:24:17.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5e97d35-aa16-4d99-abc7-c898768b0a11 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74982025-03-04T03:24:17.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1841 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5e97d35-aa16-4d99-abc7-c898768b0a11 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
74992025-03-04T03:24:17.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
75002025-03-04T03:24:17.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75012025-03-04T03:24:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75022025-03-04T03:24:17.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
75032025-03-04T03:24:17.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75042025-03-04T03:24:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75052025-03-04T03:24:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75062025-03-04T03:24:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75072025-03-04T03:24:17.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75082025-03-04T03:24:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75092025-03-04T03:24:17.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75102025-03-04T03:24:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75112025-03-04T03:24:17.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75122025-03-04T03:24:17.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
75132025-03-04T03:24:17.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75142025-03-04T03:24:17.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75152025-03-04T03:24:17.227ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
75162025-03-04T03:24:17.227ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.487828ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
75172025-03-04T03:24:17.241ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75182025-03-04T03:24:17.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75192025-03-04T03:24:17.243ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.253319ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
75202025-03-04T03:24:17.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 486 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = a7a71d3e-2f4f-4d95-a856-c42f8738fb41 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
75212025-03-04T03:24:18.116ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 80e5b4a6-dc1e-4a0a-88a4-e5aa7f90ad1d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75222025-03-04T03:24:18.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 80e5b4a6-dc1e-4a0a-88a4-e5aa7f90ad1d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75232025-03-04T03:24:18.118ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1806 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 80e5b4a6-dc1e-4a0a-88a4-e5aa7f90ad1d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75242025-03-04T03:24:19.138ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0e37784b-c0fb-44a7-884a-55d2e3546f05 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75252025-03-04T03:24:19.139ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0e37784b-c0fb-44a7-884a-55d2e3546f05 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75262025-03-04T03:24:19.141ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2521 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0e37784b-c0fb-44a7-884a-55d2e3546f05 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75272025-03-04T03:24:20.161ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 35f17d19-c685-4a5f-8f51-8a4be5a9ff39 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75282025-03-04T03:24:20.161ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 35f17d19-c685-4a5f-8f51-8a4be5a9ff39 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75292025-03-04T03:24:20.163ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2225 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 35f17d19-c685-4a5f-8f51-8a4be5a9ff39 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75302025-03-04T03:24:21.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da43d704-4390-4aa8-823f-1b2160cbe480 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75312025-03-04T03:24:21.183ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da43d704-4390-4aa8-823f-1b2160cbe480 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75322025-03-04T03:24:21.185ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2210 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da43d704-4390-4aa8-823f-1b2160cbe480 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75332025-03-04T03:24:22.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4d0f7aac-aaf0-470e-b421-2a461e965d93 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75342025-03-04T03:24:22.204ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4d0f7aac-aaf0-470e-b421-2a461e965d93 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75352025-03-04T03:24:22.206ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2571 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4d0f7aac-aaf0-470e-b421-2a461e965d93 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75362025-03-04T03:24:23.224ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ededb4f0-746d-447d-9052-9271cc5cc993 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75372025-03-04T03:24:23.225ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ededb4f0-746d-447d-9052-9271cc5cc993 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75382025-03-04T03:24:23.226ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1595 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ededb4f0-746d-447d-9052-9271cc5cc993 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75392025-03-04T03:24:24.244ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7ad5401b-f2be-43f0-afc0-7663c0f6b6f9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75402025-03-04T03:24:24.245ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7ad5401b-f2be-43f0-afc0-7663c0f6b6f9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75412025-03-04T03:24:24.247ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7ad5401b-f2be-43f0-afc0-7663c0f6b6f9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75422025-03-04T03:24:25.266ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5838402d-95ca-41f7-9f0a-1b04415816db roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75432025-03-04T03:24:25.266ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5838402d-95ca-41f7-9f0a-1b04415816db resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75442025-03-04T03:24:25.268ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2341 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5838402d-95ca-41f7-9f0a-1b04415816db response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75452025-03-04T03:24:26.288ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d68eaea7-cd77-45b6-b12f-b9b79a786a55 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75462025-03-04T03:24:26.288ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d68eaea7-cd77-45b6-b12f-b9b79a786a55 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75472025-03-04T03:24:26.290ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2127 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d68eaea7-cd77-45b6-b12f-b9b79a786a55 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75482025-03-04T03:24:27.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
75492025-03-04T03:24:27.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75502025-03-04T03:24:27.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75512025-03-04T03:24:27.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
75522025-03-04T03:24:27.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75532025-03-04T03:24:27.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75542025-03-04T03:24:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75552025-03-04T03:24:27.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75562025-03-04T03:24:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75572025-03-04T03:24:27.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75582025-03-04T03:24:27.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75592025-03-04T03:24:27.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75602025-03-04T03:24:27.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.836367ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
75612025-03-04T03:24:27.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75622025-03-04T03:24:27.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75632025-03-04T03:24:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75642025-03-04T03:24:27.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
75652025-03-04T03:24:27.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75662025-03-04T03:24:27.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75672025-03-04T03:24:27.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
75682025-03-04T03:24:27.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.380054ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
75692025-03-04T03:24:27.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f5b1760e-ab4e-428e-a531-5fcb35ba754b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75702025-03-04T03:24:27.309ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f5b1760e-ab4e-428e-a531-5fcb35ba754b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75712025-03-04T03:24:27.311ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2199 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f5b1760e-ab4e-428e-a531-5fcb35ba754b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75722025-03-04T03:24:27.895ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 470 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 292bd2a1-7a51-4583-93e6-0cc2acf4e907 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
75732025-03-04T03:24:28.330ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 18a274ec-d89c-4d4b-9964-22b6b8d88f98 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75742025-03-04T03:24:28.331ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 18a274ec-d89c-4d4b-9964-22b6b8d88f98 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75752025-03-04T03:24:28.332ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2174 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 18a274ec-d89c-4d4b-9964-22b6b8d88f98 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75762025-03-04T03:24:29.352ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a9c3616c-4f54-4a84-bf13-acf202a76689 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75772025-03-04T03:24:29.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a9c3616c-4f54-4a84-bf13-acf202a76689 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75782025-03-04T03:24:29.354ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2091 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a9c3616c-4f54-4a84-bf13-acf202a76689 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75792025-03-04T03:24:30.373ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8bb379bc-fa99-4629-8e36-b23e4674d981 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75802025-03-04T03:24:30.374ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8bb379bc-fa99-4629-8e36-b23e4674d981 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75812025-03-04T03:24:30.375ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2166 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8bb379bc-fa99-4629-8e36-b23e4674d981 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75822025-03-04T03:24:31.396ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9d94bd5-ea70-45a4-9cf6-c139fa2ecab8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75832025-03-04T03:24:31.396ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9d94bd5-ea70-45a4-9cf6-c139fa2ecab8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75842025-03-04T03:24:31.398ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2130 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9d94bd5-ea70-45a4-9cf6-c139fa2ecab8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75852025-03-04T03:24:32.417ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a57964e7-31f7-4f8c-a395-3173110c74c3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75862025-03-04T03:24:32.418ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a57964e7-31f7-4f8c-a395-3173110c74c3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75872025-03-04T03:24:32.420ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2530 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a57964e7-31f7-4f8c-a395-3173110c74c3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75882025-03-04T03:24:32.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 4e025521-255d-404e-b32d-a806d0d85db3 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
75892025-03-04T03:24:32.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 4e025521-255d-404e-b32d-a806d0d85db3 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
75902025-03-04T03:24:32.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2418 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = 4e025521-255d-404e-b32d-a806d0d85db3 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
75912025-03-04T03:24:32.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = cf4b8b6d-5bbc-4926-990c-d340f36382fa roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
75922025-03-04T03:24:32.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = cf4b8b6d-5bbc-4926-990c-d340f36382fa resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
75932025-03-04T03:24:32.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2440 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39552 req_id = cf4b8b6d-5bbc-4926-990c-d340f36382fa response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
75942025-03-04T03:24:33.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dc303342-e459-49e6-a744-e9f441800e7c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75952025-03-04T03:24:33.441ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dc303342-e459-49e6-a744-e9f441800e7c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75962025-03-04T03:24:33.443ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2315 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dc303342-e459-49e6-a744-e9f441800e7c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75972025-03-04T03:24:34.462ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 86f7b24c-a9eb-4f64-89e7-59bcfd9cb257 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75982025-03-04T03:24:34.462ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 86f7b24c-a9eb-4f64-89e7-59bcfd9cb257 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
75992025-03-04T03:24:34.464ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2553 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 86f7b24c-a9eb-4f64-89e7-59bcfd9cb257 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
76002025-03-04T03:24:35.483ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da038e1a-e3ec-481e-b74b-26711cee9dad roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
76012025-03-04T03:24:35.484ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da038e1a-e3ec-481e-b74b-26711cee9dad resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
76022025-03-04T03:24:35.485ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1996 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = da038e1a-e3ec-481e-b74b-26711cee9dad response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
76032025-03-04T03:24:36.504ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b50c46b-e2da-48db-927d-8aa3d47925df roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
76042025-03-04T03:24:36.504ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b50c46b-e2da-48db-927d-8aa3d47925df resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
76052025-03-04T03:24:36.506ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2186 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b50c46b-e2da-48db-927d-8aa3d47925df response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
76062025-03-04T03:24:37.203ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
76072025-03-04T03:24:37.203ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
76082025-03-04T03:24:37.203ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
76092025-03-04T03:24:37.203ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
76102025-03-04T03:24:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
76112025-03-04T03:24:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
76122025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
76132025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
76142025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76152025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
76162025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76172025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76182025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
76192025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76202025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Timeout
76212025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
76222025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76232025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76242025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
76252025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
76262025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76272025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76282025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
76292025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76302025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76312025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76322025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76332025-03-04T03:24:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76342025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
76352025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76362025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
76372025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
76382025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
76392025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
76402025-03-04T03:24:37.205ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
76412025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
76422025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76432025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76442025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
76452025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76462025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76472025-03-04T03:24:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
76482025-03-04T03:24:37.205ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1389, tv_nsec: 402269277 } })
76492025-03-04T03:24:37.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1389, tv_nsec: 402425759 } })
76502025-03-04T03:24:37.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
76512025-03-04T03:24:37.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76522025-03-04T03:24:37.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
76532025-03-04T03:24:37.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76542025-03-04T03:24:37.207ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76552025-03-04T03:24:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
76562025-03-04T03:24:37.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76572025-03-04T03:24:37.208ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.866348ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
76582025-03-04T03:24:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76592025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
76602025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
76612025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
76622025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
76632025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
76642025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
76652025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
76662025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76672025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
76682025-03-04T03:24:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
76692025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76702025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
76712025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
76722025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76732025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
76742025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
76752025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
76762025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76772025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76782025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76792025-03-04T03:24:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76802025-03-04T03:24:37.214ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76812025-03-04T03:24:37.215ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76822025-03-04T03:24:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76832025-03-04T03:24:37.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76842025-03-04T03:24:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76852025-03-04T03:24:37.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76862025-03-04T03:24:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76872025-03-04T03:24:37.217ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
76882025-03-04T03:24:37.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76892025-03-04T03:24:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76902025-03-04T03:24:37.219ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1389, tv_nsec: 416171678 } })
76912025-03-04T03:24:37.220ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
76922025-03-04T03:24:37.221ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76932025-03-04T03:24:37.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76942025-03-04T03:24:37.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76952025-03-04T03:24:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76962025-03-04T03:24:37.223ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76972025-03-04T03:24:37.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
76982025-03-04T03:24:37.225ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76992025-03-04T03:24:37.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77002025-03-04T03:24:37.250ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
77012025-03-04T03:24:37.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
77022025-03-04T03:24:37.256ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 46.269797ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
77032025-03-04T03:24:37.261ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77042025-03-04T03:24:37.264ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.122323ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
77052025-03-04T03:24:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
77062025-03-04T03:24:37.265ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
77072025-03-04T03:24:37.265ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77082025-03-04T03:24:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
77092025-03-04T03:24:37.266ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77102025-03-04T03:24:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77112025-03-04T03:24:37.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77122025-03-04T03:24:37.267ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
77132025-03-04T03:24:37.267ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77142025-03-04T03:24:37.267ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
77152025-03-04T03:24:37.267ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "34d87b29-146c-49ad-ad41-c4ccffa3e334", "content-length": "26", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
77162025-03-04T03:24:37.267ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77172025-03-04T03:24:37.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
77182025-03-04T03:24:37.268ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
77192025-03-04T03:24:37.268ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
77202025-03-04T03:24:37.268ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
77212025-03-04T03:24:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
77222025-03-04T03:24:37.268ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77232025-03-04T03:24:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77242025-03-04T03:24:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
77252025-03-04T03:24:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
77262025-03-04T03:24:37.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77272025-03-04T03:24:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77282025-03-04T03:24:37.269ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
77292025-03-04T03:24:37.269ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
77302025-03-04T03:24:37.269ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77312025-03-04T03:24:37.270ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cd58d1c-bdaa-4362-a773-d97b6763ad39", "content-length": "26", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
77322025-03-04T03:24:37.270ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77332025-03-04T03:24:37.270ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 60.465732ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
77342025-03-04T03:24:37.271ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
77352025-03-04T03:24:37.271ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77362025-03-04T03:24:37.271ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77372025-03-04T03:24:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
77382025-03-04T03:24:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77392025-03-04T03:24:37.271ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77402025-03-04T03:24:37.271ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
77412025-03-04T03:24:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
77422025-03-04T03:24:37.273ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
77432025-03-04T03:24:37.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
77442025-03-04T03:24:37.275ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
77452025-03-04T03:24:37.276ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 70.588074ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
77462025-03-04T03:24:37.276ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
77472025-03-04T03:24:37.276ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77482025-03-04T03:24:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77492025-03-04T03:24:37.277ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.866361ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77502025-03-04T03:24:37.281ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
77512025-03-04T03:24:37.281ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77522025-03-04T03:24:37.282ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c00ed2be-81e1-459b-addc-b18005421f19", "content-length": "26", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
77532025-03-04T03:24:37.282ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77542025-03-04T03:24:37.283ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
77552025-03-04T03:24:37.284ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
77562025-03-04T03:24:37.284ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 79.96255ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
77572025-03-04T03:24:37.285ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 80.920552ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
77582025-03-04T03:24:37.298ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77592025-03-04T03:24:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77602025-03-04T03:24:37.300ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77612025-03-04T03:24:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77622025-03-04T03:24:37.302ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77632025-03-04T03:24:37.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
77642025-03-04T03:24:37.304ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77652025-03-04T03:24:37.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77662025-03-04T03:24:37.306ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 102.476905ms iteration = 14 status = 0
77672025-03-04T03:24:37.306ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 96.957208ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
77682025-03-04T03:24:37.307ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77692025-03-04T03:24:37.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77702025-03-04T03:24:37.307ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
77712025-03-04T03:24:37.307ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
77722025-03-04T03:24:37.307ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
77732025-03-04T03:24:37.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
77742025-03-04T03:24:37.316ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
77752025-03-04T03:24:37.316ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 112.688201ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
77762025-03-04T03:24:37.326ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77772025-03-04T03:24:37.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77782025-03-04T03:24:37.334ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77792025-03-04T03:24:37.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77802025-03-04T03:24:37.336ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77812025-03-04T03:24:37.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
77822025-03-04T03:24:37.338ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77832025-03-04T03:24:37.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77842025-03-04T03:24:37.340ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 130.373488ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
77852025-03-04T03:24:37.348ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
77862025-03-04T03:24:37.348ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
77872025-03-04T03:24:37.348ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 143.258206ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
77882025-03-04T03:24:37.350ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
77892025-03-04T03:24:37.350ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b3c0638-3f84-47ce-96ba-3a6929374d1f", "content-length": "1", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
77902025-03-04T03:24:37.350ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 147.66788ms iteration = 14 status = {}
77912025-03-04T03:24:37.351ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
77922025-03-04T03:24:37.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
77932025-03-04T03:24:37.354ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 150.773438ms iteration = 14 status = null
77942025-03-04T03:24:37.354ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 150.14033ms iteration = 16 status = {"physical_disks_added":0}
77952025-03-04T03:24:37.368ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77962025-03-04T03:24:37.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
77972025-03-04T03:24:37.370ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77982025-03-04T03:24:37.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77992025-03-04T03:24:37.411ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
78002025-03-04T03:24:37.411ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
78012025-03-04T03:24:37.411ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "00d2fa47-b864-447c-ac66-daff9dec022b", "content-length": "372", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78022025-03-04T03:24:37.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
78032025-03-04T03:24:37.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78042025-03-04T03:24:37.425ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
78052025-03-04T03:24:37.425ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
78062025-03-04T03:24:37.425ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4d0bbccb-9691-4d3e-bd76-c7739de678e1", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78072025-03-04T03:24:37.425ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
78082025-03-04T03:24:37.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78092025-03-04T03:24:37.428ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
78102025-03-04T03:24:37.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78112025-03-04T03:24:37.431ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
78122025-03-04T03:24:37.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78132025-03-04T03:24:37.434ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
78142025-03-04T03:24:37.434ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
78152025-03-04T03:24:37.435ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ebd4e01a-aa14-4e18-b814-f87e66423b0d", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78162025-03-04T03:24:37.437ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 231.871421ms iteration = 14 status = {}
78172025-03-04T03:24:37.464ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78182025-03-04T03:24:37.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78192025-03-04T03:24:37.476ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
78202025-03-04T03:24:37.477ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "72c25a89-3f3d-433d-b118-f9ed313ca136", "content-length": "691", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78212025-03-04T03:24:37.477ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
78222025-03-04T03:24:37.477ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
78232025-03-04T03:24:37.477ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
78242025-03-04T03:24:37.477ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 273.600843ms iteration = 14 status = {}
78252025-03-04T03:24:37.478ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
78262025-03-04T03:24:37.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78272025-03-04T03:24:37.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78282025-03-04T03:24:37.489ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
78292025-03-04T03:24:37.489ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "40fde786-8915-4a9f-adb4-48161c1d5b65", "content-length": "134", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78302025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
78312025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
78322025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
78332025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
78342025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
78352025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
78362025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
78372025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
78382025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
78392025-03-04T03:24:37.489ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
78402025-03-04T03:24:37.490ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
78412025-03-04T03:24:37.490ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78422025-03-04T03:24:37.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78432025-03-04T03:24:37.492ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
78442025-03-04T03:24:37.492ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "349d96f2-10fe-4c9a-b52c-8cd160b6d8d2", "content-length": "2", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78452025-03-04T03:24:37.492ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
78462025-03-04T03:24:37.492ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "125adc8f-844f-415e-9c66-0d09e0f03157", "content-length": "51", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78472025-03-04T03:24:37.492ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
78482025-03-04T03:24:37.492ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
78492025-03-04T03:24:37.588ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f021992c-7a35-42ab-9ba0-17d1dc56e17d", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78502025-03-04T03:24:37.588ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
78512025-03-04T03:24:37.588ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
78522025-03-04T03:24:37.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0e28becf-2d03-4d0d-b499-91ec103b20e3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78532025-03-04T03:24:37.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0e28becf-2d03-4d0d-b499-91ec103b20e3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78542025-03-04T03:24:37.588ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "72261daa-7889-429e-8005-2e92d01950da", "content-length": "682", "date": "Tue, 04 Mar 2025 03:24:37 GMT"} })
78552025-03-04T03:24:37.588ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78562025-03-04T03:24:37.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78572025-03-04T03:24:37.590ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2398 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0e28becf-2d03-4d0d-b499-91ec103b20e3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78582025-03-04T03:24:37.592ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78592025-03-04T03:24:37.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78602025-03-04T03:24:37.595ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78612025-03-04T03:24:37.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78622025-03-04T03:24:37.597ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78632025-03-04T03:24:37.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78642025-03-04T03:24:37.604ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 400.648881ms iteration = 14 status = {}
78652025-03-04T03:24:37.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 512 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = a9d1d346-9920-4c14-94d2-fee469e79046 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
78662025-03-04T03:24:38.611ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88cbdd2e-22ad-40ca-898c-1f95114fdceb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78672025-03-04T03:24:38.612ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88cbdd2e-22ad-40ca-898c-1f95114fdceb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78682025-03-04T03:24:38.614ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88cbdd2e-22ad-40ca-898c-1f95114fdceb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78692025-03-04T03:24:39.633ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f588304a-4cf4-4edd-8416-734721ee621a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78702025-03-04T03:24:39.633ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f588304a-4cf4-4edd-8416-734721ee621a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78712025-03-04T03:24:39.635ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2423 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f588304a-4cf4-4edd-8416-734721ee621a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78722025-03-04T03:24:40.652ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6bc9ab1-3cc7-42d1-838a-3785b7546189 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78732025-03-04T03:24:40.653ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6bc9ab1-3cc7-42d1-838a-3785b7546189 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78742025-03-04T03:24:40.655ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2276 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6bc9ab1-3cc7-42d1-838a-3785b7546189 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78752025-03-04T03:24:41.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7d2faace-7477-47ab-a900-4b3683d4eb15 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78762025-03-04T03:24:41.674ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7d2faace-7477-47ab-a900-4b3683d4eb15 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78772025-03-04T03:24:41.676ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2462 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7d2faace-7477-47ab-a900-4b3683d4eb15 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78782025-03-04T03:24:42.695ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0622845c-ac97-4c0a-8a89-3a2736dd9332 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78792025-03-04T03:24:42.695ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0622845c-ac97-4c0a-8a89-3a2736dd9332 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78802025-03-04T03:24:42.697ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1881 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0622845c-ac97-4c0a-8a89-3a2736dd9332 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78812025-03-04T03:24:43.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1579484-dd24-46dd-831f-bae7f28ebc88 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78822025-03-04T03:24:43.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1579484-dd24-46dd-831f-bae7f28ebc88 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78832025-03-04T03:24:43.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2338 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1579484-dd24-46dd-831f-bae7f28ebc88 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78842025-03-04T03:24:44.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1b7b8fdd-50fa-4dd9-b98b-659144494d44 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78852025-03-04T03:24:44.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1b7b8fdd-50fa-4dd9-b98b-659144494d44 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78862025-03-04T03:24:44.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2151 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1b7b8fdd-50fa-4dd9-b98b-659144494d44 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78872025-03-04T03:24:45.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4c7bdf15-5ab0-4688-bf58-0eaed6a7c271 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78882025-03-04T03:24:45.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4c7bdf15-5ab0-4688-bf58-0eaed6a7c271 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78892025-03-04T03:24:45.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2633 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4c7bdf15-5ab0-4688-bf58-0eaed6a7c271 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78902025-03-04T03:24:46.779ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 572388f1-94d2-453e-8edf-18a1de8624ba roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78912025-03-04T03:24:46.779ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 572388f1-94d2-453e-8edf-18a1de8624ba resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78922025-03-04T03:24:46.781ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2959 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 572388f1-94d2-453e-8edf-18a1de8624ba response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
78932025-03-04T03:24:47.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
78942025-03-04T03:24:47.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78952025-03-04T03:24:47.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78962025-03-04T03:24:47.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
78972025-03-04T03:24:47.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78982025-03-04T03:24:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78992025-03-04T03:24:47.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79002025-03-04T03:24:47.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79012025-03-04T03:24:47.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79022025-03-04T03:24:47.208ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.368991ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
79032025-03-04T03:24:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79042025-03-04T03:24:47.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79052025-03-04T03:24:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79062025-03-04T03:24:47.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79072025-03-04T03:24:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79082025-03-04T03:24:47.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79092025-03-04T03:24:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
79102025-03-04T03:24:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79112025-03-04T03:24:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79122025-03-04T03:24:47.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
79132025-03-04T03:24:47.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.759086ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
79142025-03-04T03:24:47.800ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e4706464-b0ed-416f-ab2c-cbb313845493 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79152025-03-04T03:24:47.800ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e4706464-b0ed-416f-ab2c-cbb313845493 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79162025-03-04T03:24:47.802ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2477 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e4706464-b0ed-416f-ab2c-cbb313845493 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79172025-03-04T03:24:47.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 468 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 97747684-76eb-40ed-9533-ac36138fd67e response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
79182025-03-04T03:24:48.822ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2edab419-f3f4-48fa-95df-ebd343163a61 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79192025-03-04T03:24:48.822ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2edab419-f3f4-48fa-95df-ebd343163a61 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79202025-03-04T03:24:48.824ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1993 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2edab419-f3f4-48fa-95df-ebd343163a61 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79212025-03-04T03:24:49.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 77ee3b3e-3a49-4226-ae06-99a87ad9b1da roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79222025-03-04T03:24:49.844ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 77ee3b3e-3a49-4226-ae06-99a87ad9b1da resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79232025-03-04T03:24:49.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2197 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 77ee3b3e-3a49-4226-ae06-99a87ad9b1da response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79242025-03-04T03:24:50.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b171eb72-c16f-42ac-a90c-2a99ffc16bfb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79252025-03-04T03:24:50.866ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b171eb72-c16f-42ac-a90c-2a99ffc16bfb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79262025-03-04T03:24:50.867ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2232 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b171eb72-c16f-42ac-a90c-2a99ffc16bfb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79272025-03-04T03:24:51.886ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae65bf7b-4245-42c1-b052-992c6a67ff71 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79282025-03-04T03:24:51.887ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae65bf7b-4245-42c1-b052-992c6a67ff71 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79292025-03-04T03:24:51.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2051 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae65bf7b-4245-42c1-b052-992c6a67ff71 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79302025-03-04T03:24:52.908ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7afd49ac-2edd-4ab4-bb75-834fbec1dea4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79312025-03-04T03:24:52.908ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7afd49ac-2edd-4ab4-bb75-834fbec1dea4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79322025-03-04T03:24:52.910ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2073 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7afd49ac-2edd-4ab4-bb75-834fbec1dea4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79332025-03-04T03:24:53.930ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 204f64d6-5988-45fb-bc9a-ebd627c41c82 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79342025-03-04T03:24:53.931ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 204f64d6-5988-45fb-bc9a-ebd627c41c82 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79352025-03-04T03:24:53.932ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2063 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 204f64d6-5988-45fb-bc9a-ebd627c41c82 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79362025-03-04T03:24:54.951ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a088db3b-213f-4291-8f48-f48fe02bc71b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79372025-03-04T03:24:54.951ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a088db3b-213f-4291-8f48-f48fe02bc71b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79382025-03-04T03:24:54.953ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2196 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a088db3b-213f-4291-8f48-f48fe02bc71b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79392025-03-04T03:24:55.972ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6c8d61a1-4d3c-403d-a3a1-56f2d9514323 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79402025-03-04T03:24:55.972ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6c8d61a1-4d3c-403d-a3a1-56f2d9514323 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79412025-03-04T03:24:55.974ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2338 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6c8d61a1-4d3c-403d-a3a1-56f2d9514323 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79422025-03-04T03:24:56.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8ec9f2af-8344-4a5c-b407-62b140b20d27 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79432025-03-04T03:24:56.992ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8ec9f2af-8344-4a5c-b407-62b140b20d27 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79442025-03-04T03:24:56.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1851 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8ec9f2af-8344-4a5c-b407-62b140b20d27 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79452025-03-04T03:24:57.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
79462025-03-04T03:24:57.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79472025-03-04T03:24:57.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79482025-03-04T03:24:57.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
79492025-03-04T03:24:57.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79502025-03-04T03:24:57.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79512025-03-04T03:24:57.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79522025-03-04T03:24:57.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79532025-03-04T03:24:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79542025-03-04T03:24:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79552025-03-04T03:24:57.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79562025-03-04T03:24:57.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.812466ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
79572025-03-04T03:24:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79582025-03-04T03:24:57.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79592025-03-04T03:24:57.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79602025-03-04T03:24:57.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79612025-03-04T03:24:57.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
79622025-03-04T03:24:57.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79632025-03-04T03:24:57.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79642025-03-04T03:24:57.227ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
79652025-03-04T03:24:57.227ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.905371ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
79662025-03-04T03:24:57.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 524 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 2ad25b24-f392-4fd8-a931-39b68fff3852 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
79672025-03-04T03:24:58.013ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6e0e4e0c-486b-40bd-a838-2bc08f0b3bde roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79682025-03-04T03:24:58.013ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6e0e4e0c-486b-40bd-a838-2bc08f0b3bde resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79692025-03-04T03:24:58.015ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1985 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6e0e4e0c-486b-40bd-a838-2bc08f0b3bde response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79702025-03-04T03:24:59.034ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e8abe5c5-418b-4cd9-b5d5-f09dc07a7188 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79712025-03-04T03:24:59.034ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e8abe5c5-418b-4cd9-b5d5-f09dc07a7188 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79722025-03-04T03:24:59.036ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2148 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e8abe5c5-418b-4cd9-b5d5-f09dc07a7188 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79732025-03-04T03:25:00.057ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f99cb8d0-ab56-473e-86a6-123ac2eb14db roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79742025-03-04T03:25:00.057ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f99cb8d0-ab56-473e-86a6-123ac2eb14db resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79752025-03-04T03:25:00.059ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2147 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f99cb8d0-ab56-473e-86a6-123ac2eb14db response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79762025-03-04T03:25:01.078ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 159e8325-cecd-481b-a376-300c437e513b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79772025-03-04T03:25:01.078ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 159e8325-cecd-481b-a376-300c437e513b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79782025-03-04T03:25:01.080ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1922 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 159e8325-cecd-481b-a376-300c437e513b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79792025-03-04T03:25:02.099ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 611c213c-6dc6-4e71-babb-ad383e3f8844 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79802025-03-04T03:25:02.099ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 611c213c-6dc6-4e71-babb-ad383e3f8844 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79812025-03-04T03:25:02.101ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2277 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 611c213c-6dc6-4e71-babb-ad383e3f8844 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79822025-03-04T03:25:03.120ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84c4f7ea-bb77-478a-83ff-db493c74ee4d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79832025-03-04T03:25:03.120ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84c4f7ea-bb77-478a-83ff-db493c74ee4d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79842025-03-04T03:25:03.124ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3945 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 84c4f7ea-bb77-478a-83ff-db493c74ee4d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79852025-03-04T03:25:04.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f1cedbb4-82ee-43cc-b669-757ddfcff1ae roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79862025-03-04T03:25:04.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f1cedbb4-82ee-43cc-b669-757ddfcff1ae resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79872025-03-04T03:25:04.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2633 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f1cedbb4-82ee-43cc-b669-757ddfcff1ae response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79882025-03-04T03:25:05.172ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2022f459-2f39-4c63-a3f9-f4fbe68d57ef roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79892025-03-04T03:25:05.172ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2022f459-2f39-4c63-a3f9-f4fbe68d57ef resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79902025-03-04T03:25:05.174ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2057 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2022f459-2f39-4c63-a3f9-f4fbe68d57ef response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79912025-03-04T03:25:06.193ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c6045af-fb9e-4498-91bc-d0e5c0ca410b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79922025-03-04T03:25:06.194ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c6045af-fb9e-4498-91bc-d0e5c0ca410b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79932025-03-04T03:25:06.195ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1893 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c6045af-fb9e-4498-91bc-d0e5c0ca410b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
79942025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
79952025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
79962025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
79972025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79982025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
79992025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
80002025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
80012025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
80022025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
80032025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
80042025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
80052025-03-04T03:25:07.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80062025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
80072025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
80082025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
80092025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
80102025-03-04T03:25:07.204ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:15:07.203976829 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
80112025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
80122025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
80132025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
80142025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
80152025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
80162025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
80172025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
80182025-03-04T03:25:07.204ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
80192025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
80202025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.061\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
80212025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
80222025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
80232025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80242025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
80252025-03-04T03:25:07.204ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
80262025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80272025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
80282025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
80292025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
80302025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
80312025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80322025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80332025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80342025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80352025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
80362025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80372025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
80382025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80392025-03-04T03:25:07.204ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1419, tv_nsec: 400773601 } })
80402025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
80412025-03-04T03:25:07.204ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1419, tv_nsec: 400880192 } })
80422025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
80432025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
80442025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80452025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
80462025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80472025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80482025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80492025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80502025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
80512025-03-04T03:25:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.030432ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
80522025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
80532025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.123494ms iteration = 8 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
80542025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
80552025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
80562025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80572025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
80582025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80592025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
80602025-03-04T03:25:07.205ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
80612025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80622025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
80632025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
80642025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
80652025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
80662025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80672025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80682025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
80692025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
80702025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80712025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80722025-03-04T03:25:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80732025-03-04T03:25:07.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
80742025-03-04T03:25:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80752025-03-04T03:25:07.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1419, tv_nsec: 403135760 } })
80762025-03-04T03:25:07.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
80772025-03-04T03:25:07.207ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
80782025-03-04T03:25:07.207ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
80792025-03-04T03:25:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
80802025-03-04T03:25:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.32447ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
80812025-03-04T03:25:07.207ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80822025-03-04T03:25:07.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1419, tv_nsec: 404112952 } })
80832025-03-04T03:25:07.208ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
80842025-03-04T03:25:07.208ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:15:07.203976829 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
80852025-03-04T03:25:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.86758ms iteration = 8 status = {"expiration":"2025-03-04T03:15:07.203976829Z","pruned":[]}
80862025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80872025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
80882025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80892025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80902025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
80912025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
80922025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
80932025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
80942025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
80952025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
80962025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
80972025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
80982025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
80992025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
81002025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
81012025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
81022025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
81032025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
81042025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
81052025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
81062025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81072025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
81082025-03-04T03:25:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
81092025-03-04T03:25:07.211ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81102025-03-04T03:25:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
81112025-03-04T03:25:07.211ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
81122025-03-04T03:25:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
81132025-03-04T03:25:07.211ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81142025-03-04T03:25:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
81152025-03-04T03:25:07.212ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81162025-03-04T03:25:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81172025-03-04T03:25:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
81182025-03-04T03:25:07.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
81192025-03-04T03:25:07.213ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81202025-03-04T03:25:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
81212025-03-04T03:25:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
81222025-03-04T03:25:07.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81232025-03-04T03:25:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81242025-03-04T03:25:07.214ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ddab408a-bcc9-4bd9-9765-316e1a51666b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
81252025-03-04T03:25:07.214ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ddab408a-bcc9-4bd9-9765-316e1a51666b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
81262025-03-04T03:25:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
81272025-03-04T03:25:07.216ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81282025-03-04T03:25:07.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81292025-03-04T03:25:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81302025-03-04T03:25:07.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81312025-03-04T03:25:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
81322025-03-04T03:25:07.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
81332025-03-04T03:25:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
81342025-03-04T03:25:07.218ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81352025-03-04T03:25:07.218ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81362025-03-04T03:25:07.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
81372025-03-04T03:25:07.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81382025-03-04T03:25:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81392025-03-04T03:25:07.220ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
81402025-03-04T03:25:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
81412025-03-04T03:25:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
81422025-03-04T03:25:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
81432025-03-04T03:25:07.225ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 15.765293ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
81442025-03-04T03:25:07.228ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
81452025-03-04T03:25:07.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
81462025-03-04T03:25:07.229ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81472025-03-04T03:25:07.230ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.597093ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
81482025-03-04T03:25:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81492025-03-04T03:25:07.230ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 16606 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ddab408a-bcc9-4bd9-9765-316e1a51666b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
81502025-03-04T03:25:07.231ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81512025-03-04T03:25:07.231ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1419, tv_nsec: 427591700 } })
81522025-03-04T03:25:07.231ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
81532025-03-04T03:25:07.233ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81542025-03-04T03:25:07.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81552025-03-04T03:25:07.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
81562025-03-04T03:25:07.235ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
81572025-03-04T03:25:07.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
81582025-03-04T03:25:07.243ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81592025-03-04T03:25:07.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
81602025-03-04T03:25:07.246ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
81612025-03-04T03:25:07.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
81622025-03-04T03:25:07.260ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
81632025-03-04T03:25:07.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
81642025-03-04T03:25:07.265ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81652025-03-04T03:25:07.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81662025-03-04T03:25:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81672025-03-04T03:25:07.266ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
81682025-03-04T03:25:07.266ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
81692025-03-04T03:25:07.266ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
81702025-03-04T03:25:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
81712025-03-04T03:25:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
81722025-03-04T03:25:07.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81732025-03-04T03:25:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81742025-03-04T03:25:07.270ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81752025-03-04T03:25:07.270ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
81762025-03-04T03:25:07.270ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
81772025-03-04T03:25:07.270ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
81782025-03-04T03:25:07.270ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81792025-03-04T03:25:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81802025-03-04T03:25:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81812025-03-04T03:25:07.272ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
81822025-03-04T03:25:07.272ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
81832025-03-04T03:25:07.272ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 64.736013ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
81842025-03-04T03:25:07.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 68.574711ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
81852025-03-04T03:25:07.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81862025-03-04T03:25:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81872025-03-04T03:25:07.275ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 65.936058ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
81882025-03-04T03:25:07.275ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
81892025-03-04T03:25:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
81902025-03-04T03:25:07.275ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
81912025-03-04T03:25:07.275ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
81922025-03-04T03:25:07.275ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
81932025-03-04T03:25:07.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
81942025-03-04T03:25:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
81952025-03-04T03:25:07.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81962025-03-04T03:25:07.277ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 67.71802ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
81972025-03-04T03:25:07.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81982025-03-04T03:25:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81992025-03-04T03:25:07.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82002025-03-04T03:25:07.278ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
82012025-03-04T03:25:07.278ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
82022025-03-04T03:25:07.278ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 74.549673ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
82032025-03-04T03:25:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
82042025-03-04T03:25:07.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82052025-03-04T03:25:07.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82062025-03-04T03:25:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82072025-03-04T03:25:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82082025-03-04T03:25:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
82092025-03-04T03:25:07.282ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
82102025-03-04T03:25:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
82112025-03-04T03:25:07.283ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
82122025-03-04T03:25:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
82132025-03-04T03:25:07.284ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82142025-03-04T03:25:07.284ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
82152025-03-04T03:25:07.285ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
82162025-03-04T03:25:07.285ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
82172025-03-04T03:25:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
82182025-03-04T03:25:07.285ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82192025-03-04T03:25:07.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82202025-03-04T03:25:07.286ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82212025-03-04T03:25:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82222025-03-04T03:25:07.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
82232025-03-04T03:25:07.287ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82242025-03-04T03:25:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82252025-03-04T03:25:07.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82262025-03-04T03:25:07.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
82272025-03-04T03:25:07.289ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82282025-03-04T03:25:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82292025-03-04T03:25:07.291ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82302025-03-04T03:25:07.291ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
82312025-03-04T03:25:07.291ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 86.034534ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
82322025-03-04T03:25:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
82332025-03-04T03:25:07.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
82342025-03-04T03:25:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
82352025-03-04T03:25:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
82362025-03-04T03:25:07.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82372025-03-04T03:25:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82382025-03-04T03:25:07.294ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82392025-03-04T03:25:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
82402025-03-04T03:25:07.294ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
82412025-03-04T03:25:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
82422025-03-04T03:25:07.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
82432025-03-04T03:25:07.295ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82442025-03-04T03:25:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82452025-03-04T03:25:07.298ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
82462025-03-04T03:25:07.298ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 94.341816ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
82472025-03-04T03:25:07.298ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 94.496148ms iteration = 15 status = 0
82482025-03-04T03:25:07.299ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
82492025-03-04T03:25:07.299ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
82502025-03-04T03:25:07.299ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
82512025-03-04T03:25:07.299ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 95.318538ms iteration = 8 status = {"generation":1}
82522025-03-04T03:25:07.302ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
82532025-03-04T03:25:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
82542025-03-04T03:25:07.303ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
82552025-03-04T03:25:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
82562025-03-04T03:25:07.304ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82572025-03-04T03:25:07.305ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
82582025-03-04T03:25:07.305ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 100.940237ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
82592025-03-04T03:25:07.306ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
82602025-03-04T03:25:07.306ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
82612025-03-04T03:25:07.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
82622025-03-04T03:25:07.307ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 102.510227ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
82632025-03-04T03:25:07.308ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82642025-03-04T03:25:07.308ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
82652025-03-04T03:25:07.308ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
82662025-03-04T03:25:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
82672025-03-04T03:25:07.313ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82682025-03-04T03:25:07.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 104.780834ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
82692025-03-04T03:25:07.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
82702025-03-04T03:25:07.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
82712025-03-04T03:25:07.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
82722025-03-04T03:25:07.331ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
82732025-03-04T03:25:07.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
82742025-03-04T03:25:07.342ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82752025-03-04T03:25:07.342ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
82762025-03-04T03:25:07.342ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 137.386084ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
82772025-03-04T03:25:07.346ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
82782025-03-04T03:25:07.346ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82792025-03-04T03:25:07.346ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8c1c8ed-4028-4ace-833d-56a1de9ea229", "content-length": "26", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
82802025-03-04T03:25:07.346ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82812025-03-04T03:25:07.348ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
82822025-03-04T03:25:07.350ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
82832025-03-04T03:25:07.350ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 147.064402ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
82842025-03-04T03:25:07.352ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
82852025-03-04T03:25:07.352ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
82862025-03-04T03:25:07.352ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
82872025-03-04T03:25:07.352ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
82882025-03-04T03:25:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
82892025-03-04T03:25:07.360ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82902025-03-04T03:25:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
82912025-03-04T03:25:07.362ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
82922025-03-04T03:25:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
82932025-03-04T03:25:07.363ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
82942025-03-04T03:25:07.363ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82952025-03-04T03:25:07.364ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2981e9ba-f6d6-4d4c-85cd-6fbf190ec84f", "content-length": "26", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
82962025-03-04T03:25:07.364ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82972025-03-04T03:25:07.365ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
82982025-03-04T03:25:07.365ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
82992025-03-04T03:25:07.365ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
83002025-03-04T03:25:07.365ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 161.616321ms iteration = 8 status = {"generation":2}
83012025-03-04T03:25:07.365ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
83022025-03-04T03:25:07.372ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 162.650503ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
83032025-03-04T03:25:07.410ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 205.701151ms iteration = 17 status = {"physical_disks_added":0}
83042025-03-04T03:25:07.410ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
83052025-03-04T03:25:07.410ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 205.66669ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
83062025-03-04T03:25:07.463ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
83072025-03-04T03:25:07.523ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
83082025-03-04T03:25:07.523ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "53ea3836-11f9-4d8a-9755-fdccedbd7f9a", "content-length": "1", "date": "Tue, 04 Mar 2025 03:25:06 GMT"} })
83092025-03-04T03:25:07.524ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
83102025-03-04T03:25:07.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
83112025-03-04T03:25:07.526ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 322.943518ms iteration = 15 status = null
83122025-03-04T03:25:07.527ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
83132025-03-04T03:25:07.530ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
83142025-03-04T03:25:07.530ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
83152025-03-04T03:25:07.530ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "db93dce2-2dbf-4628-8c44-c100bd558769", "content-length": "372", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
83162025-03-04T03:25:07.530ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83172025-03-04T03:25:07.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83182025-03-04T03:25:07.541ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
83192025-03-04T03:25:07.541ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83202025-03-04T03:25:07.541ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "48717607-112a-4e84-b1ac-c71ec8ceb513", "content-length": "26", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
83212025-03-04T03:25:07.541ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83222025-03-04T03:25:07.544ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
83232025-03-04T03:25:07.544ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
83242025-03-04T03:25:07.544ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
83252025-03-04T03:25:07.544ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "cc079289-0d35-4482-a49d-5ea2039e01f7", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
83262025-03-04T03:25:07.544ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83272025-03-04T03:25:07.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83282025-03-04T03:25:07.547ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83292025-03-04T03:25:07.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83302025-03-04T03:25:07.549ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 345.267132ms iteration = 15 status = {}
83312025-03-04T03:25:07.550ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83322025-03-04T03:25:07.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83332025-03-04T03:25:07.551ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
83342025-03-04T03:25:07.552ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "76fe4848-331d-4f3c-8581-ec9e6a482dd4", "content-length": "691", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
83352025-03-04T03:25:07.552ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
83362025-03-04T03:25:07.552ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
83372025-03-04T03:25:07.552ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
83382025-03-04T03:25:07.553ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 349.517963ms iteration = 15 status = {}
83392025-03-04T03:25:07.555ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
83402025-03-04T03:25:07.555ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
83412025-03-04T03:25:07.555ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b1ec336a-d5d2-41cd-a3cd-d05373ac82a3", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
83422025-03-04T03:25:07.556ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 351.544148ms iteration = 15 status = {}
83432025-03-04T03:25:07.599ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
83442025-03-04T03:25:07.599ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "dbee70f5-d94b-4307-9cb1-b3a67ff167f9", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
83452025-03-04T03:25:07.599ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
83462025-03-04T03:25:07.601ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "4471e617-0501-4090-9838-b9d568ffa760", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
83472025-03-04T03:25:07.601ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
83482025-03-04T03:25:07.603ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
83492025-03-04T03:25:07.603ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 399.204792ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
83502025-03-04T03:25:07.611ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
83512025-03-04T03:25:07.611ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "06f9ccaf-cdcd-49ad-9113-c01bf2b52e5a", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
83522025-03-04T03:25:07.611ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
83532025-03-04T03:25:07.613ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e605bf39-20a5-4243-8843-2958d73a9591", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
83542025-03-04T03:25:07.613ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
83552025-03-04T03:25:07.615ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "e6fe0d29-01c8-4c75-880a-ff0b77171e14", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
83562025-03-04T03:25:07.615ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
83572025-03-04T03:25:07.616ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
83582025-03-04T03:25:07.616ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 413.141074ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
83592025-03-04T03:25:07.672ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83602025-03-04T03:25:07.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83612025-03-04T03:25:07.675ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
83622025-03-04T03:25:07.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83632025-03-04T03:25:07.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83642025-03-04T03:25:07.683ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
83652025-03-04T03:25:07.683ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "aec01d84-7fa0-49ac-a19f-ae128e5c173e", "content-length": "134", "date": "Tue, 04 Mar 2025 03:25:06 GMT"} })
83662025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
83672025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
83682025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
83692025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
83702025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83712025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83722025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83732025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
83742025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83752025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
83762025-03-04T03:25:07.683ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
83772025-03-04T03:25:07.683ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83782025-03-04T03:25:07.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83792025-03-04T03:25:07.685ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
83802025-03-04T03:25:07.685ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ecba6ae-c4bd-4bb3-a2f7-0ad128177b9b", "content-length": "2", "date": "Tue, 04 Mar 2025 03:25:06 GMT"} })
83812025-03-04T03:25:07.685ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
83822025-03-04T03:25:07.685ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f414c039-8c5e-40cb-a04b-41f168837e25", "content-length": "51", "date": "Tue, 04 Mar 2025 03:25:06 GMT"} })
83832025-03-04T03:25:07.686ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
83842025-03-04T03:25:07.686ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
83852025-03-04T03:25:07.776ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "617110b0-c0b9-4c00-bd5a-36cb6e575a99", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
83862025-03-04T03:25:07.776ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
83872025-03-04T03:25:07.776ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
83882025-03-04T03:25:07.779ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8ce2d61-a4a4-4d8a-baa2-f0cb6659b769", "content-length": "682", "date": "Tue, 04 Mar 2025 03:25:07 GMT"} })
83892025-03-04T03:25:07.779ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83902025-03-04T03:25:07.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83912025-03-04T03:25:07.783ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83922025-03-04T03:25:07.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83932025-03-04T03:25:07.787ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83942025-03-04T03:25:07.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83952025-03-04T03:25:07.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83962025-03-04T03:25:07.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83972025-03-04T03:25:07.800ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 596.604323ms iteration = 15 status = {}
83982025-03-04T03:25:07.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 489 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 91db21ba-bbea-41c7-9e2b-e5ae9b12a5ee response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
83992025-03-04T03:25:08.261ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edd61ae6-9b2c-4ce5-94a0-c60c4ee57ea8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84002025-03-04T03:25:08.261ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edd61ae6-9b2c-4ce5-94a0-c60c4ee57ea8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84012025-03-04T03:25:08.262ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1613 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edd61ae6-9b2c-4ce5-94a0-c60c4ee57ea8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84022025-03-04T03:25:09.281ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 77f35d75-df3b-4681-a276-c195809c61df roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84032025-03-04T03:25:09.282ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 77f35d75-df3b-4681-a276-c195809c61df resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84042025-03-04T03:25:09.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2459 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 77f35d75-df3b-4681-a276-c195809c61df response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84052025-03-04T03:25:10.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6db2841d-4383-4e43-928a-86e083e6cf98 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84062025-03-04T03:25:10.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6db2841d-4383-4e43-928a-86e083e6cf98 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84072025-03-04T03:25:10.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2303 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6db2841d-4383-4e43-928a-86e083e6cf98 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84082025-03-04T03:25:11.324ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eed29bdf-4f21-4f88-af06-ef469d8acb99 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84092025-03-04T03:25:11.324ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eed29bdf-4f21-4f88-af06-ef469d8acb99 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84102025-03-04T03:25:11.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2310 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eed29bdf-4f21-4f88-af06-ef469d8acb99 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84112025-03-04T03:25:12.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c265758f-e07e-42e2-bfa9-dc36f3920cbc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84122025-03-04T03:25:12.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c265758f-e07e-42e2-bfa9-dc36f3920cbc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84132025-03-04T03:25:12.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2514 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c265758f-e07e-42e2-bfa9-dc36f3920cbc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84142025-03-04T03:25:13.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3126a4c-9f85-40e7-8d4b-78d81bd1b083 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84152025-03-04T03:25:13.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3126a4c-9f85-40e7-8d4b-78d81bd1b083 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84162025-03-04T03:25:13.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2226 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3126a4c-9f85-40e7-8d4b-78d81bd1b083 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84172025-03-04T03:25:14.388ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c5ffca53-eae7-4458-81f4-bb07baa7da12 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84182025-03-04T03:25:14.388ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c5ffca53-eae7-4458-81f4-bb07baa7da12 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84192025-03-04T03:25:14.390ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2290 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c5ffca53-eae7-4458-81f4-bb07baa7da12 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84202025-03-04T03:25:15.411ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de8160f7-aef8-4184-bc87-d298f0f712cf roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84212025-03-04T03:25:15.411ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de8160f7-aef8-4184-bc87-d298f0f712cf resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84222025-03-04T03:25:15.413ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2802 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de8160f7-aef8-4184-bc87-d298f0f712cf response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84232025-03-04T03:25:16.433ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7ba1fd4-7710-4b5c-9c11-acce1e24670e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84242025-03-04T03:25:16.434ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7ba1fd4-7710-4b5c-9c11-acce1e24670e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84252025-03-04T03:25:16.435ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2052 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7ba1fd4-7710-4b5c-9c11-acce1e24670e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84262025-03-04T03:25:17.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
84272025-03-04T03:25:17.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84282025-03-04T03:25:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84292025-03-04T03:25:17.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
84302025-03-04T03:25:17.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84312025-03-04T03:25:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84322025-03-04T03:25:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84332025-03-04T03:25:17.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84342025-03-04T03:25:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84352025-03-04T03:25:17.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.796996ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
84362025-03-04T03:25:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
84372025-03-04T03:25:17.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84382025-03-04T03:25:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84392025-03-04T03:25:17.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84402025-03-04T03:25:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84412025-03-04T03:25:17.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84422025-03-04T03:25:17.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
84432025-03-04T03:25:17.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84442025-03-04T03:25:17.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84452025-03-04T03:25:17.224ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
84462025-03-04T03:25:17.224ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.842783ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
84472025-03-04T03:25:17.454ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1cde9720-6260-4e76-ad0c-6bbea76f0c07 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84482025-03-04T03:25:17.454ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1cde9720-6260-4e76-ad0c-6bbea76f0c07 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84492025-03-04T03:25:17.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1667 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1cde9720-6260-4e76-ad0c-6bbea76f0c07 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84502025-03-04T03:25:17.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 588 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 2ee63d4e-1055-443b-8507-2654a4d323e4 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
84512025-03-04T03:25:18.474ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 62ee1e5f-fd63-416b-9770-578a5895a92d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84522025-03-04T03:25:18.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 62ee1e5f-fd63-416b-9770-578a5895a92d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84532025-03-04T03:25:18.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1945 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 62ee1e5f-fd63-416b-9770-578a5895a92d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84542025-03-04T03:25:19.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7e8582a-7729-4a2c-9c98-094cbb5a85e7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84552025-03-04T03:25:19.496ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7e8582a-7729-4a2c-9c98-094cbb5a85e7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84562025-03-04T03:25:19.497ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2156 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7e8582a-7729-4a2c-9c98-094cbb5a85e7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84572025-03-04T03:25:20.517ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42fc6b51-bb98-4370-a931-b2f50a90bd1c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84582025-03-04T03:25:20.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42fc6b51-bb98-4370-a931-b2f50a90bd1c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84592025-03-04T03:25:20.519ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2052 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42fc6b51-bb98-4370-a931-b2f50a90bd1c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84602025-03-04T03:25:21.538ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a33757e8-f5e7-48e5-bfb6-241cc416726a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84612025-03-04T03:25:21.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a33757e8-f5e7-48e5-bfb6-241cc416726a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84622025-03-04T03:25:21.540ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2039 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a33757e8-f5e7-48e5-bfb6-241cc416726a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84632025-03-04T03:25:22.559ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 66e40ede-bb13-4631-9dcf-eb9872b8a98a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84642025-03-04T03:25:22.559ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 66e40ede-bb13-4631-9dcf-eb9872b8a98a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84652025-03-04T03:25:22.561ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2082 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 66e40ede-bb13-4631-9dcf-eb9872b8a98a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84662025-03-04T03:25:23.581ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9028da46-99f5-427e-bb3f-482dc19feedd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84672025-03-04T03:25:23.581ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9028da46-99f5-427e-bb3f-482dc19feedd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84682025-03-04T03:25:23.583ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2490 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9028da46-99f5-427e-bb3f-482dc19feedd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84692025-03-04T03:25:24.602ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c68c167b-0079-40d4-854d-ccd22d3cb35b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84702025-03-04T03:25:24.602ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c68c167b-0079-40d4-854d-ccd22d3cb35b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84712025-03-04T03:25:24.604ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2519 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c68c167b-0079-40d4-854d-ccd22d3cb35b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84722025-03-04T03:25:25.623ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a2ff115-5133-4862-9a23-cd4f28952642 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84732025-03-04T03:25:25.624ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a2ff115-5133-4862-9a23-cd4f28952642 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84742025-03-04T03:25:25.625ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2006 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a2ff115-5133-4862-9a23-cd4f28952642 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84752025-03-04T03:25:26.645ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 968d63da-bcfd-4a32-ae11-69680722721b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84762025-03-04T03:25:26.646ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 968d63da-bcfd-4a32-ae11-69680722721b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84772025-03-04T03:25:26.648ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2236 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 968d63da-bcfd-4a32-ae11-69680722721b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
84782025-03-04T03:25:27.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
84792025-03-04T03:25:27.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84802025-03-04T03:25:27.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84812025-03-04T03:25:27.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
84822025-03-04T03:25:27.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84832025-03-04T03:25:27.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84842025-03-04T03:25:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84852025-03-04T03:25:27.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84862025-03-04T03:25:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84872025-03-04T03:25:27.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
84882025-03-04T03:25:27.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84892025-03-04T03:25:27.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.061509ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
84902025-03-04T03:25:27.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84912025-03-04T03:25:27.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84922025-03-04T03:25:27.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84932025-03-04T03:25:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84942025-03-04T03:25:27.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
84952025-03-04T03:25:27.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84962025-03-04T03:25:27.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84972025-03-04T03:25:27.224ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
84982025-03-04T03:25:27.224ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.468679ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
84992025-03-04T03:25:27.666ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7bdc94d0-61b3-4f5a-aacb-6cecce0d293a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85002025-03-04T03:25:27.667ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7bdc94d0-61b3-4f5a-aacb-6cecce0d293a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85012025-03-04T03:25:27.669ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2303 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7bdc94d0-61b3-4f5a-aacb-6cecce0d293a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85022025-03-04T03:25:27.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 478 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 851a878c-41ea-4749-96b0-80dfca47e3aa response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
85032025-03-04T03:25:28.688ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 696af057-95ef-473e-8669-8c34593571a0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85042025-03-04T03:25:28.688ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 696af057-95ef-473e-8669-8c34593571a0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85052025-03-04T03:25:28.690ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2445 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 696af057-95ef-473e-8669-8c34593571a0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85062025-03-04T03:25:29.710ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 96339f0d-3e6a-4559-971f-f13dd28db616 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85072025-03-04T03:25:29.710ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 96339f0d-3e6a-4559-971f-f13dd28db616 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85082025-03-04T03:25:29.712ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1924 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 96339f0d-3e6a-4559-971f-f13dd28db616 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85092025-03-04T03:25:30.731ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5366a122-3893-43b4-82ef-2e10da7ed607 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85102025-03-04T03:25:30.731ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5366a122-3893-43b4-82ef-2e10da7ed607 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85112025-03-04T03:25:30.733ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2023 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5366a122-3893-43b4-82ef-2e10da7ed607 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85122025-03-04T03:25:31.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cb619c1c-a493-4d5b-ba37-bdfb9036abca roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85132025-03-04T03:25:31.753ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cb619c1c-a493-4d5b-ba37-bdfb9036abca resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85142025-03-04T03:25:31.754ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1968 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cb619c1c-a493-4d5b-ba37-bdfb9036abca response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85152025-03-04T03:25:32.774ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 578e1121-b08b-4bb1-b589-a7e40616281c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85162025-03-04T03:25:32.775ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 578e1121-b08b-4bb1-b589-a7e40616281c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85172025-03-04T03:25:32.777ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2610 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 578e1121-b08b-4bb1-b589-a7e40616281c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85182025-03-04T03:25:32.890ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:44097
85192025-03-04T03:25:32.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 7060b9fe-5081-41b1-9b30-23fddb301e4d roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
85202025-03-04T03:25:32.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 7060b9fe-5081-41b1-9b30-23fddb301e4d resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
85212025-03-04T03:25:32.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2897 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 7060b9fe-5081-41b1-9b30-23fddb301e4d response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
85222025-03-04T03:25:32.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 55469931-633b-408e-be32-8af99fa4950a roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
85232025-03-04T03:25:32.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 55469931-633b-408e-be32-8af99fa4950a resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
85242025-03-04T03:25:32.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2338 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 55469931-633b-408e-be32-8af99fa4950a response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
85252025-03-04T03:25:33.796ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 293341a9-1c7e-43d9-a502-7a5dbedb3300 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85262025-03-04T03:25:33.796ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 293341a9-1c7e-43d9-a502-7a5dbedb3300 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85272025-03-04T03:25:33.798ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1939 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 293341a9-1c7e-43d9-a502-7a5dbedb3300 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85282025-03-04T03:25:34.817ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edc7afa8-4221-4c24-a733-f21958b095bc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85292025-03-04T03:25:34.818ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edc7afa8-4221-4c24-a733-f21958b095bc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85302025-03-04T03:25:34.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2003 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edc7afa8-4221-4c24-a733-f21958b095bc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85312025-03-04T03:25:35.839ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed5384aa-df68-4dc3-a26c-b859b112cb58 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85322025-03-04T03:25:35.839ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed5384aa-df68-4dc3-a26c-b859b112cb58 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85332025-03-04T03:25:35.841ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2072 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ed5384aa-df68-4dc3-a26c-b859b112cb58 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85342025-03-04T03:25:36.860ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec41218c-18d0-463c-970a-1ed2da26aced roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85352025-03-04T03:25:36.860ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec41218c-18d0-463c-970a-1ed2da26aced resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85362025-03-04T03:25:36.862ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2433 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec41218c-18d0-463c-970a-1ed2da26aced response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
85372025-03-04T03:25:37.203ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
85382025-03-04T03:25:37.203ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
85392025-03-04T03:25:37.203ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
85402025-03-04T03:25:37.203ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
85412025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
85422025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
85432025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
85442025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
85452025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
85462025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
85472025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
85482025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85492025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85502025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
85512025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
85522025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
85532025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
85542025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
85552025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85562025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
85572025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85582025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85592025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85602025-03-04T03:25:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
85612025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
85622025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85632025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85642025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
85652025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
85662025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
85672025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
85682025-03-04T03:25:37.205ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
85692025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
85702025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
85712025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85722025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
85732025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
85742025-03-04T03:25:37.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
85752025-03-04T03:25:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85762025-03-04T03:25:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
85772025-03-04T03:25:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85782025-03-04T03:25:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
85792025-03-04T03:25:37.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1449, tv_nsec: 402265701 } })
85802025-03-04T03:25:37.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1449, tv_nsec: 402362192 } })
85812025-03-04T03:25:37.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
85822025-03-04T03:25:37.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
85832025-03-04T03:25:37.207ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85842025-03-04T03:25:37.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85852025-03-04T03:25:37.208ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85862025-03-04T03:25:37.208ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.212082ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
85872025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
85882025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85892025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85902025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
85912025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
85922025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
85932025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
85942025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
85952025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
85962025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
85972025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
85982025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
85992025-03-04T03:25:37.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
86002025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
86012025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
86022025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
86032025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
86042025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
86052025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
86062025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
86072025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
86082025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
86092025-03-04T03:25:37.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86102025-03-04T03:25:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86112025-03-04T03:25:37.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86122025-03-04T03:25:37.214ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86132025-03-04T03:25:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
86142025-03-04T03:25:37.214ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
86152025-03-04T03:25:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
86162025-03-04T03:25:37.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86172025-03-04T03:25:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
86182025-03-04T03:25:37.216ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
86192025-03-04T03:25:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
86202025-03-04T03:25:37.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86212025-03-04T03:25:37.216ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86222025-03-04T03:25:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86232025-03-04T03:25:37.216ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
86242025-03-04T03:25:37.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86252025-03-04T03:25:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86262025-03-04T03:25:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
86272025-03-04T03:25:37.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
86282025-03-04T03:25:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
86292025-03-04T03:25:37.218ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86302025-03-04T03:25:37.218ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
86312025-03-04T03:25:37.218ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
86322025-03-04T03:25:37.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
86332025-03-04T03:25:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86342025-03-04T03:25:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
86352025-03-04T03:25:37.219ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
86362025-03-04T03:25:37.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
86372025-03-04T03:25:37.220ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 10.492658ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
86382025-03-04T03:25:37.220ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
86392025-03-04T03:25:37.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
86402025-03-04T03:25:37.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86412025-03-04T03:25:37.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86422025-03-04T03:25:37.222ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
86432025-03-04T03:25:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
86442025-03-04T03:25:37.222ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86452025-03-04T03:25:37.222ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1449, tv_nsec: 418832614 } })
86462025-03-04T03:25:37.223ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
86472025-03-04T03:25:37.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
86482025-03-04T03:25:37.224ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86492025-03-04T03:25:37.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86502025-03-04T03:25:37.226ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86512025-03-04T03:25:37.228ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
86522025-03-04T03:25:37.228ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 22.230353ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
86532025-03-04T03:25:37.228ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
86542025-03-04T03:25:37.228ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
86552025-03-04T03:25:37.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
86562025-03-04T03:25:37.243ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86572025-03-04T03:25:37.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86582025-03-04T03:25:37.249ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86592025-03-04T03:25:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86602025-03-04T03:25:37.252ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86612025-03-04T03:25:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
86622025-03-04T03:25:37.254ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86632025-03-04T03:25:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86642025-03-04T03:25:37.257ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 52.492484ms iteration = 16 status = 0
86652025-03-04T03:25:37.257ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
86662025-03-04T03:25:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
86672025-03-04T03:25:37.259ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 53.562577ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
86682025-03-04T03:25:37.259ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
86692025-03-04T03:25:37.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
86702025-03-04T03:25:37.264ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 55.060834ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
86712025-03-04T03:25:37.267ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
86722025-03-04T03:25:37.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86732025-03-04T03:25:37.268ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 58.380135ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
86742025-03-04T03:25:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
86752025-03-04T03:25:37.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
86762025-03-04T03:25:37.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86772025-03-04T03:25:37.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86782025-03-04T03:25:37.272ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86792025-03-04T03:25:37.274ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
86802025-03-04T03:25:37.274ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
86812025-03-04T03:25:37.274ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
86822025-03-04T03:25:37.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
86832025-03-04T03:25:37.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
86842025-03-04T03:25:37.275ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
86852025-03-04T03:25:37.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
86862025-03-04T03:25:37.276ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
86872025-03-04T03:25:37.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
86882025-03-04T03:25:37.281ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
86892025-03-04T03:25:37.281ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
86902025-03-04T03:25:37.284ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "baf50353-8c62-44f6-b061-eca5d39ea7b9", "content-length": "26", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
86912025-03-04T03:25:37.284ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
86922025-03-04T03:25:37.284ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
86932025-03-04T03:25:37.284ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 79.480005ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
86942025-03-04T03:25:37.286ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
86952025-03-04T03:25:37.287ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
86962025-03-04T03:25:37.287ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
86972025-03-04T03:25:37.287ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "28d3d27c-2552-46d8-a631-db61c01fb40c", "content-length": "26", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
86982025-03-04T03:25:37.287ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
86992025-03-04T03:25:37.289ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
87002025-03-04T03:25:37.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 93.252173ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
87012025-03-04T03:25:37.300ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 90.145385ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
87022025-03-04T03:25:37.303ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 93.616157ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
87032025-03-04T03:25:37.311ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
87042025-03-04T03:25:37.311ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
87052025-03-04T03:25:37.311ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 105.516484ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
87062025-03-04T03:25:37.312ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
87072025-03-04T03:25:37.312ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
87082025-03-04T03:25:37.312ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9ca5fdb-9156-4d25-8407-2fa566a2bb5c", "content-length": "26", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87092025-03-04T03:25:37.312ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
87102025-03-04T03:25:37.312ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
87112025-03-04T03:25:37.312ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 108.282277ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
87122025-03-04T03:25:37.314ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
87132025-03-04T03:25:37.336ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87142025-03-04T03:25:37.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87152025-03-04T03:25:37.338ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87162025-03-04T03:25:37.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
87172025-03-04T03:25:37.339ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87182025-03-04T03:25:37.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87192025-03-04T03:25:37.341ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87202025-03-04T03:25:37.343ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
87212025-03-04T03:25:37.343ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87222025-03-04T03:25:37.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87232025-03-04T03:25:37.355ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 150.864939ms iteration = 18 status = {"physical_disks_added":0}
87242025-03-04T03:25:37.381ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 177.582036ms iteration = 16 status = {}
87252025-03-04T03:25:37.389ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
87262025-03-04T03:25:37.390ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "745c0534-f25a-4bef-9709-7dcb78c3253a", "content-length": "1", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87272025-03-04T03:25:37.391ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
87282025-03-04T03:25:37.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
87292025-03-04T03:25:37.393ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 190.026809ms iteration = 16 status = null
87302025-03-04T03:25:37.425ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
87312025-03-04T03:25:37.425ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
87322025-03-04T03:25:37.426ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "84e505e9-8060-4155-8ccb-2b1473e1078e", "content-length": "372", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87332025-03-04T03:25:37.426ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87342025-03-04T03:25:37.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87352025-03-04T03:25:37.440ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
87362025-03-04T03:25:37.440ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
87372025-03-04T03:25:37.441ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5e5aa58a-97dc-472a-be13-67b0b80cdc8c", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87382025-03-04T03:25:37.441ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87392025-03-04T03:25:37.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87402025-03-04T03:25:37.472ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
87412025-03-04T03:25:37.472ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa5c2e33-23a1-4ebe-94e3-63e3fa0c8b8c", "content-length": "691", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87422025-03-04T03:25:37.472ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
87432025-03-04T03:25:37.472ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
87442025-03-04T03:25:37.472ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
87452025-03-04T03:25:37.474ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 269.374391ms iteration = 16 status = {}
87462025-03-04T03:25:37.474ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87472025-03-04T03:25:37.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87482025-03-04T03:25:37.477ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87492025-03-04T03:25:37.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87502025-03-04T03:25:37.480ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
87512025-03-04T03:25:37.480ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
87522025-03-04T03:25:37.480ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "095a0915-09a9-4116-b341-76c58835bbdc", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87532025-03-04T03:25:37.482ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 277.181047ms iteration = 16 status = {}
87542025-03-04T03:25:37.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87552025-03-04T03:25:37.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87562025-03-04T03:25:37.506ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
87572025-03-04T03:25:37.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87582025-03-04T03:25:37.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87592025-03-04T03:25:37.513ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
87602025-03-04T03:25:37.513ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b4bf10a-d1f5-4ddf-ad56-6119a63c2504", "content-length": "134", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87612025-03-04T03:25:37.513ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
87622025-03-04T03:25:37.513ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
87632025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
87642025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
87652025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
87662025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
87672025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
87682025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
87692025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
87702025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
87712025-03-04T03:25:37.514ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
87722025-03-04T03:25:37.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87732025-03-04T03:25:37.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87742025-03-04T03:25:37.516ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
87752025-03-04T03:25:37.516ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "31758c05-2e05-438f-a90f-c6bc0967d8eb", "content-length": "2", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87762025-03-04T03:25:37.516ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
87772025-03-04T03:25:37.516ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f53c948b-f6cf-4c4b-86ce-a7c64b199b7b", "content-length": "51", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87782025-03-04T03:25:37.516ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
87792025-03-04T03:25:37.516ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
87802025-03-04T03:25:37.612ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0836158e-705d-4205-8b93-82532ae911ff", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87812025-03-04T03:25:37.612ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
87822025-03-04T03:25:37.612ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
87832025-03-04T03:25:37.613ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ccd530b-6bdf-4243-8887-e3e9d03857db", "content-length": "682", "date": "Tue, 04 Mar 2025 03:25:37 GMT"} })
87842025-03-04T03:25:37.613ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87852025-03-04T03:25:37.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87862025-03-04T03:25:37.617ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87872025-03-04T03:25:37.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87882025-03-04T03:25:37.620ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87892025-03-04T03:25:37.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87902025-03-04T03:25:37.622ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87912025-03-04T03:25:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87922025-03-04T03:25:37.629ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 424.505742ms iteration = 16 status = {}
87932025-03-04T03:25:37.881ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d195ab90-2f9c-413f-8910-8c8624efdbc7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
87942025-03-04T03:25:37.882ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d195ab90-2f9c-413f-8910-8c8624efdbc7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
87952025-03-04T03:25:37.883ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1775 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d195ab90-2f9c-413f-8910-8c8624efdbc7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
87962025-03-04T03:25:37.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 558 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 471ade0c-2cbf-4316-8e4b-62015d77e84a response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
87972025-03-04T03:25:38.937ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5eba1264-1427-4dca-8c9f-355302b28b04 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
87982025-03-04T03:25:38.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5eba1264-1427-4dca-8c9f-355302b28b04 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
87992025-03-04T03:25:38.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2564 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5eba1264-1427-4dca-8c9f-355302b28b04 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88002025-03-04T03:25:39.963ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21270a3d-857e-4e2f-925b-5d6b4afdab87 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88012025-03-04T03:25:39.963ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21270a3d-857e-4e2f-925b-5d6b4afdab87 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88022025-03-04T03:25:39.965ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2212 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21270a3d-857e-4e2f-925b-5d6b4afdab87 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88032025-03-04T03:25:40.983ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f27642b1-9512-42a5-b303-52627bbe0059 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88042025-03-04T03:25:40.984ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f27642b1-9512-42a5-b303-52627bbe0059 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88052025-03-04T03:25:40.986ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2424 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f27642b1-9512-42a5-b303-52627bbe0059 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88062025-03-04T03:25:42.004ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 30e1099f-19ea-42e2-8207-eae74f79be21 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88072025-03-04T03:25:42.004ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 30e1099f-19ea-42e2-8207-eae74f79be21 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88082025-03-04T03:25:42.006ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2495 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 30e1099f-19ea-42e2-8207-eae74f79be21 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88092025-03-04T03:25:43.025ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 042a5ec6-87ff-4e0c-902b-ae5aa1ba6d8b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88102025-03-04T03:25:43.026ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 042a5ec6-87ff-4e0c-902b-ae5aa1ba6d8b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88112025-03-04T03:25:43.028ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2192 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 042a5ec6-87ff-4e0c-902b-ae5aa1ba6d8b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88122025-03-04T03:25:44.047ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 45ae161a-6cc4-478b-a787-62d31ce080b1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88132025-03-04T03:25:44.047ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 45ae161a-6cc4-478b-a787-62d31ce080b1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88142025-03-04T03:25:44.049ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2251 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 45ae161a-6cc4-478b-a787-62d31ce080b1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88152025-03-04T03:25:45.069ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f51e82e-3ce9-432c-a95d-e9c0b397f68e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88162025-03-04T03:25:45.069ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f51e82e-3ce9-432c-a95d-e9c0b397f68e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88172025-03-04T03:25:45.071ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2785 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f51e82e-3ce9-432c-a95d-e9c0b397f68e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88182025-03-04T03:25:46.091ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0672b27-ce91-4583-8f2a-a752d9763320 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88192025-03-04T03:25:46.092ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0672b27-ce91-4583-8f2a-a752d9763320 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88202025-03-04T03:25:46.093ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1920 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0672b27-ce91-4583-8f2a-a752d9763320 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88212025-03-04T03:25:47.112ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4399c17c-4a07-438a-90a6-74888289043a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88222025-03-04T03:25:47.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4399c17c-4a07-438a-90a6-74888289043a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88232025-03-04T03:25:47.115ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2700 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4399c17c-4a07-438a-90a6-74888289043a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88242025-03-04T03:25:47.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
88252025-03-04T03:25:47.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88262025-03-04T03:25:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88272025-03-04T03:25:47.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
88282025-03-04T03:25:47.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88292025-03-04T03:25:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88302025-03-04T03:25:47.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88312025-03-04T03:25:47.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88322025-03-04T03:25:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88332025-03-04T03:25:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
88342025-03-04T03:25:47.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88352025-03-04T03:25:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88362025-03-04T03:25:47.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.00669ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
88372025-03-04T03:25:47.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88382025-03-04T03:25:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88392025-03-04T03:25:47.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88402025-03-04T03:25:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
88412025-03-04T03:25:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88422025-03-04T03:25:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88432025-03-04T03:25:47.224ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
88442025-03-04T03:25:47.224ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.858311ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
88452025-03-04T03:25:47.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 477 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 33d94b18-cbe2-4408-9b40-65105f614365 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
88462025-03-04T03:25:48.134ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ddfc988f-363b-40d6-b67b-d878841ef8e2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88472025-03-04T03:25:48.134ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ddfc988f-363b-40d6-b67b-d878841ef8e2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88482025-03-04T03:25:48.136ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1978 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ddfc988f-363b-40d6-b67b-d878841ef8e2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88492025-03-04T03:25:49.155ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1f5c1d62-a6d2-4231-aa9f-ced1c6101b0a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88502025-03-04T03:25:49.155ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1f5c1d62-a6d2-4231-aa9f-ced1c6101b0a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88512025-03-04T03:25:49.157ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1921 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1f5c1d62-a6d2-4231-aa9f-ced1c6101b0a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88522025-03-04T03:25:50.175ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec46a4be-1193-4c5d-aa07-806da2563056 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88532025-03-04T03:25:50.175ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec46a4be-1193-4c5d-aa07-806da2563056 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88542025-03-04T03:25:50.177ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2165 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec46a4be-1193-4c5d-aa07-806da2563056 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88552025-03-04T03:25:51.196ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9f463d9-0b73-485f-b5db-f52fe4bbd2fa roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88562025-03-04T03:25:51.196ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9f463d9-0b73-485f-b5db-f52fe4bbd2fa resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88572025-03-04T03:25:51.198ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1929 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e9f463d9-0b73-485f-b5db-f52fe4bbd2fa response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88582025-03-04T03:25:52.217ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a30fcfc-9942-4277-8a5c-24411442389d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88592025-03-04T03:25:52.217ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a30fcfc-9942-4277-8a5c-24411442389d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88602025-03-04T03:25:52.219ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1949 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a30fcfc-9942-4277-8a5c-24411442389d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88612025-03-04T03:25:53.238ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 33d03e84-5c58-4777-8556-6ad3ae7f523d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88622025-03-04T03:25:53.238ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 33d03e84-5c58-4777-8556-6ad3ae7f523d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88632025-03-04T03:25:53.240ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2411 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 33d03e84-5c58-4777-8556-6ad3ae7f523d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88642025-03-04T03:25:54.259ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 388406cc-6724-49d6-8303-23f61fb57fcd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88652025-03-04T03:25:54.260ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 388406cc-6724-49d6-8303-23f61fb57fcd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88662025-03-04T03:25:54.261ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2022 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 388406cc-6724-49d6-8303-23f61fb57fcd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88672025-03-04T03:25:55.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 887f25aa-597e-4de9-8742-db712516cf5f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88682025-03-04T03:25:55.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 887f25aa-597e-4de9-8742-db712516cf5f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88692025-03-04T03:25:55.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1915 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 887f25aa-597e-4de9-8742-db712516cf5f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88702025-03-04T03:25:56.301ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fbd7eaf9-c683-4e56-a8ce-5cbb67b35ae7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88712025-03-04T03:25:56.301ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fbd7eaf9-c683-4e56-a8ce-5cbb67b35ae7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88722025-03-04T03:25:56.303ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2345 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fbd7eaf9-c683-4e56-a8ce-5cbb67b35ae7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88732025-03-04T03:25:57.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
88742025-03-04T03:25:57.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88752025-03-04T03:25:57.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88762025-03-04T03:25:57.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
88772025-03-04T03:25:57.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88782025-03-04T03:25:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88792025-03-04T03:25:57.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88802025-03-04T03:25:57.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88812025-03-04T03:25:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88822025-03-04T03:25:57.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.585964ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
88832025-03-04T03:25:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
88842025-03-04T03:25:57.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88852025-03-04T03:25:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88862025-03-04T03:25:57.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88872025-03-04T03:25:57.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88882025-03-04T03:25:57.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88892025-03-04T03:25:57.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
88902025-03-04T03:25:57.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88912025-03-04T03:25:57.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88922025-03-04T03:25:57.224ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
88932025-03-04T03:25:57.224ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.776632ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
88942025-03-04T03:25:57.324ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 60e0716b-49ac-4d86-9352-8371ea27ec76 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88952025-03-04T03:25:57.324ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 60e0716b-49ac-4d86-9352-8371ea27ec76 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88962025-03-04T03:25:57.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2089 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 60e0716b-49ac-4d86-9352-8371ea27ec76 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88972025-03-04T03:25:57.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 524 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 2d7e2a5a-3cb1-4987-8ce5-06cfa488bb3d response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
88982025-03-04T03:25:58.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3c385b25-485e-4508-8bda-3346df1d707e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
88992025-03-04T03:25:58.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3c385b25-485e-4508-8bda-3346df1d707e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89002025-03-04T03:25:58.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2353 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3c385b25-485e-4508-8bda-3346df1d707e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89012025-03-04T03:25:59.368ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9055d3e2-3840-49da-9808-d9d75b6d0191 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89022025-03-04T03:25:59.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9055d3e2-3840-49da-9808-d9d75b6d0191 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89032025-03-04T03:25:59.370ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2447 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9055d3e2-3840-49da-9808-d9d75b6d0191 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89042025-03-04T03:26:00.391ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a0b17c2-9b6b-4624-866d-1fe065a641fe roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89052025-03-04T03:26:00.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a0b17c2-9b6b-4624-866d-1fe065a641fe resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89062025-03-04T03:26:00.393ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2637 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a0b17c2-9b6b-4624-866d-1fe065a641fe response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89072025-03-04T03:26:01.412ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09ae35ba-2cc8-4b88-a233-90d2963a5aae roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89082025-03-04T03:26:01.412ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09ae35ba-2cc8-4b88-a233-90d2963a5aae resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89092025-03-04T03:26:01.414ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1794 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09ae35ba-2cc8-4b88-a233-90d2963a5aae response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89102025-03-04T03:26:02.433ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8aac5d3f-274d-422d-b871-1c598704ea02 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89112025-03-04T03:26:02.433ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8aac5d3f-274d-422d-b871-1c598704ea02 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89122025-03-04T03:26:02.435ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2474 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8aac5d3f-274d-422d-b871-1c598704ea02 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89132025-03-04T03:26:02.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = b8a704c5-f155-43c6-81c4-d5a77991d8f5 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
89142025-03-04T03:26:02.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = b8a704c5-f155-43c6-81c4-d5a77991d8f5 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
89152025-03-04T03:26:02.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2299 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = b8a704c5-f155-43c6-81c4-d5a77991d8f5 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
89162025-03-04T03:26:02.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = a89db734-13c5-4760-be2d-4843c831c3fb roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
89172025-03-04T03:26:02.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = a89db734-13c5-4760-be2d-4843c831c3fb resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
89182025-03-04T03:26:02.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2041 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = a89db734-13c5-4760-be2d-4843c831c3fb response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
89192025-03-04T03:26:03.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68fe5958-cbb7-46dc-a51e-3cad21d930cb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89202025-03-04T03:26:03.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68fe5958-cbb7-46dc-a51e-3cad21d930cb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89212025-03-04T03:26:03.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2573 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68fe5958-cbb7-46dc-a51e-3cad21d930cb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89222025-03-04T03:26:04.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b9cca48e-1c24-4ba2-804f-a9945265c7b4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89232025-03-04T03:26:04.481ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b9cca48e-1c24-4ba2-804f-a9945265c7b4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89242025-03-04T03:26:04.483ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2113 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b9cca48e-1c24-4ba2-804f-a9945265c7b4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89252025-03-04T03:26:05.502ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7c52d9fd-609b-4866-bb3a-86e5dfe78d12 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89262025-03-04T03:26:05.503ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7c52d9fd-609b-4866-bb3a-86e5dfe78d12 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89272025-03-04T03:26:05.504ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2211 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7c52d9fd-609b-4866-bb3a-86e5dfe78d12 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89282025-03-04T03:26:06.525ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c033fde-7dbf-4df2-8d02-0bf874f456b2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89292025-03-04T03:26:06.525ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c033fde-7dbf-4df2-8d02-0bf874f456b2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89302025-03-04T03:26:06.527ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1972 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5c033fde-7dbf-4df2-8d02-0bf874f456b2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
89312025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
89322025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
89332025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
89342025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
89352025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
89362025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
89372025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
89382025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
89392025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
89402025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
89412025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
89422025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
89432025-03-04T03:26:07.204ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:16:07.204663969 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
89442025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
89452025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
89462025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
89472025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
89482025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
89492025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
89502025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
89512025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
89522025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
89532025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89542025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
89552025-03-04T03:26:07.204ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
89562025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 17.991\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
89572025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
89582025-03-04T03:26:07.204ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
89592025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
89602025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
89612025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
89622025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89632025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
89642025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
89652025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
89662025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Timeout
89672025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89682025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
89692025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
89702025-03-04T03:26:07.204ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89712025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
89722025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
89732025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
89742025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
89752025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
89762025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
89772025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
89782025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89792025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
89802025-03-04T03:26:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1479, tv_nsec: 401292054 } })
89812025-03-04T03:26:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1479, tv_nsec: 401294454 } })
89822025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89832025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89842025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89852025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
89862025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
89872025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
89882025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
89892025-03-04T03:26:07.205ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
89902025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
89912025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89922025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
89932025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
89942025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
89952025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
89962025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
89972025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
89982025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
89992025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90002025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
90012025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
90022025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
90032025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.208125ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
90042025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90052025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
90062025-03-04T03:26:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.372027ms iteration = 9 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
90072025-03-04T03:26:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
90082025-03-04T03:26:07.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
90092025-03-04T03:26:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
90102025-03-04T03:26:07.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
90112025-03-04T03:26:07.207ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
90122025-03-04T03:26:07.207ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
90132025-03-04T03:26:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
90142025-03-04T03:26:07.208ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:16:07.204663969 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
90152025-03-04T03:26:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.353021ms iteration = 9 status = {"expiration":"2025-03-04T03:16:07.204663969Z","pruned":[]}
90162025-03-04T03:26:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
90172025-03-04T03:26:07.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1479, tv_nsec: 404132138 } })
90182025-03-04T03:26:07.208ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
90192025-03-04T03:26:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.118241ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
90202025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
90212025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
90222025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
90232025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
90242025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
90252025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
90262025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
90272025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
90282025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
90292025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
90302025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
90312025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
90322025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90332025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
90342025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
90352025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
90362025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
90372025-03-04T03:26:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
90382025-03-04T03:26:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
90392025-03-04T03:26:07.210ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90402025-03-04T03:26:07.210ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90412025-03-04T03:26:07.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90422025-03-04T03:26:07.211ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1479, tv_nsec: 407801373 } })
90432025-03-04T03:26:07.212ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
90442025-03-04T03:26:07.212ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
90452025-03-04T03:26:07.212ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
90462025-03-04T03:26:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
90472025-03-04T03:26:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
90482025-03-04T03:26:07.212ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
90492025-03-04T03:26:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
90502025-03-04T03:26:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90512025-03-04T03:26:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
90522025-03-04T03:26:07.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
90532025-03-04T03:26:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
90542025-03-04T03:26:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
90552025-03-04T03:26:07.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
90562025-03-04T03:26:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
90572025-03-04T03:26:07.214ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90582025-03-04T03:26:07.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90592025-03-04T03:26:07.216ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
90602025-03-04T03:26:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90612025-03-04T03:26:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
90622025-03-04T03:26:07.216ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90632025-03-04T03:26:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90642025-03-04T03:26:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90652025-03-04T03:26:07.217ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90662025-03-04T03:26:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
90672025-03-04T03:26:07.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90682025-03-04T03:26:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90692025-03-04T03:26:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90702025-03-04T03:26:07.219ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90712025-03-04T03:26:07.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90722025-03-04T03:26:07.220ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90732025-03-04T03:26:07.220ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
90742025-03-04T03:26:07.220ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
90752025-03-04T03:26:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
90762025-03-04T03:26:07.220ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90772025-03-04T03:26:07.221ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90782025-03-04T03:26:07.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
90792025-03-04T03:26:07.222ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90802025-03-04T03:26:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90812025-03-04T03:26:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
90822025-03-04T03:26:07.222ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
90832025-03-04T03:26:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
90842025-03-04T03:26:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
90852025-03-04T03:26:07.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
90862025-03-04T03:26:07.226ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
90872025-03-04T03:26:07.226ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
90882025-03-04T03:26:07.226ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
90892025-03-04T03:26:07.226ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
90902025-03-04T03:26:07.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
90912025-03-04T03:26:07.227ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 17.864459ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
90922025-03-04T03:26:07.227ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.865909ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
90932025-03-04T03:26:07.227ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 17.99424ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
90942025-03-04T03:26:07.228ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 18.451006ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
90952025-03-04T03:26:07.228ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
90962025-03-04T03:26:07.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
90972025-03-04T03:26:07.229ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
90982025-03-04T03:26:07.229ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 24.346099ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
90992025-03-04T03:26:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
91002025-03-04T03:26:07.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
91012025-03-04T03:26:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
91022025-03-04T03:26:07.230ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
91032025-03-04T03:26:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
91042025-03-04T03:26:07.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91052025-03-04T03:26:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91062025-03-04T03:26:07.231ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1479, tv_nsec: 427693997 } })
91072025-03-04T03:26:07.232ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
91082025-03-04T03:26:07.232ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91092025-03-04T03:26:07.233ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91102025-03-04T03:26:07.233ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91112025-03-04T03:26:07.233ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
91122025-03-04T03:26:07.233ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
91132025-03-04T03:26:07.233ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
91142025-03-04T03:26:07.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
91152025-03-04T03:26:07.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
91162025-03-04T03:26:07.234ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91172025-03-04T03:26:07.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91182025-03-04T03:26:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
91192025-03-04T03:26:07.237ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
91202025-03-04T03:26:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
91212025-03-04T03:26:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
91222025-03-04T03:26:07.237ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
91232025-03-04T03:26:07.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
91242025-03-04T03:26:07.238ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
91252025-03-04T03:26:07.238ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 32.59805ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
91262025-03-04T03:26:07.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91272025-03-04T03:26:07.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91282025-03-04T03:26:07.243ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
91292025-03-04T03:26:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
91302025-03-04T03:26:07.243ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
91312025-03-04T03:26:07.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
91322025-03-04T03:26:07.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
91332025-03-04T03:26:07.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
91342025-03-04T03:26:07.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91352025-03-04T03:26:07.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91362025-03-04T03:26:07.253ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91372025-03-04T03:26:07.255ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
91382025-03-04T03:26:07.255ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
91392025-03-04T03:26:07.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
91402025-03-04T03:26:07.263ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
91412025-03-04T03:26:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
91422025-03-04T03:26:07.269ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
91432025-03-04T03:26:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
91442025-03-04T03:26:07.272ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
91452025-03-04T03:26:07.273ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
91462025-03-04T03:26:07.273ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
91472025-03-04T03:26:07.273ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
91482025-03-04T03:26:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
91492025-03-04T03:26:07.275ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
91502025-03-04T03:26:07.275ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
91512025-03-04T03:26:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
91522025-03-04T03:26:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
91532025-03-04T03:26:07.281ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
91542025-03-04T03:26:07.282ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
91552025-03-04T03:26:07.282ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 74.107389ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
91562025-03-04T03:26:07.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
91572025-03-04T03:26:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
91582025-03-04T03:26:07.290ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
91592025-03-04T03:26:07.290ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
91602025-03-04T03:26:07.290ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 85.978274ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
91612025-03-04T03:26:07.291ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
91622025-03-04T03:26:07.291ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
91632025-03-04T03:26:07.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
91642025-03-04T03:26:07.291ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 82.145947ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
91652025-03-04T03:26:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
91662025-03-04T03:26:07.298ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
91672025-03-04T03:26:07.298ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
91682025-03-04T03:26:07.298ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 92.745057ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
91692025-03-04T03:26:07.299ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
91702025-03-04T03:26:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
91712025-03-04T03:26:07.308ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 103.450028ms iteration = 19 status = {"physical_disks_added":0}
91722025-03-04T03:26:07.371ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
91732025-03-04T03:26:07.371ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
91742025-03-04T03:26:07.371ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d763edd-2ba3-4f27-89fe-4e424cbea4f5", "content-length": "26", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
91752025-03-04T03:26:07.371ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
91762025-03-04T03:26:07.375ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
91772025-03-04T03:26:07.376ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
91782025-03-04T03:26:07.387ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
91792025-03-04T03:26:07.387ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
91802025-03-04T03:26:07.387ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcc0ce89-5d75-4d47-9bb7-fc6e3b9ff64a", "content-length": "26", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
91812025-03-04T03:26:07.387ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
91822025-03-04T03:26:07.389ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
91832025-03-04T03:26:07.419ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
91842025-03-04T03:26:07.419ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
91852025-03-04T03:26:07.419ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcecd82d-5cf6-4bcc-89ef-cb5cf1cd3cbb", "content-length": "372", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
91862025-03-04T03:26:07.419ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91872025-03-04T03:26:07.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91882025-03-04T03:26:07.422ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 217.635208ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
91892025-03-04T03:26:07.424ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
91902025-03-04T03:26:07.424ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
91912025-03-04T03:26:07.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
91922025-03-04T03:26:07.425ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91932025-03-04T03:26:07.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
91942025-03-04T03:26:07.428ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
91952025-03-04T03:26:07.428ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
91962025-03-04T03:26:07.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
91972025-03-04T03:26:07.433ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
91982025-03-04T03:26:07.433ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
91992025-03-04T03:26:07.433ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "46b51969-c720-4529-abd7-bacf1429ceb8", "content-length": "26", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
92002025-03-04T03:26:07.433ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
92012025-03-04T03:26:07.435ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
92022025-03-04T03:26:07.435ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 231.060732ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
92032025-03-04T03:26:07.436ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
92042025-03-04T03:26:07.464ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92052025-03-04T03:26:07.466ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
92062025-03-04T03:26:07.467ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 257.549256ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
92072025-03-04T03:26:07.468ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
92082025-03-04T03:26:07.469ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
92092025-03-04T03:26:07.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
92102025-03-04T03:26:07.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92112025-03-04T03:26:07.472ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 267.413058ms iteration = 17 status = 0
92122025-03-04T03:26:07.473ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
92132025-03-04T03:26:07.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
92142025-03-04T03:26:07.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
92152025-03-04T03:26:07.475ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
92162025-03-04T03:26:07.475ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 271.078142ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
92172025-03-04T03:26:07.475ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
92182025-03-04T03:26:07.476ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
92192025-03-04T03:26:07.476ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
92202025-03-04T03:26:07.476ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 271.344006ms iteration = 9 status = {"generation":2}
92212025-03-04T03:26:07.476ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92222025-03-04T03:26:07.476ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
92232025-03-04T03:26:07.476ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 271.759151ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
92242025-03-04T03:26:07.478ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
92252025-03-04T03:26:07.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
92262025-03-04T03:26:07.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
92272025-03-04T03:26:07.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92282025-03-04T03:26:07.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92292025-03-04T03:26:07.481ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
92302025-03-04T03:26:07.481ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
92312025-03-04T03:26:07.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92322025-03-04T03:26:07.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92332025-03-04T03:26:07.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92342025-03-04T03:26:07.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
92352025-03-04T03:26:07.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92362025-03-04T03:26:07.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92372025-03-04T03:26:07.489ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
92382025-03-04T03:26:07.489ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
92392025-03-04T03:26:07.489ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
92402025-03-04T03:26:07.489ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 285.194566ms iteration = 9 status = {"generation":1}
92412025-03-04T03:26:07.493ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
92422025-03-04T03:26:07.493ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
92432025-03-04T03:26:07.493ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3551c9fd-bc47-4b04-8e6c-e9336ee53747", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
92442025-03-04T03:26:07.493ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
92452025-03-04T03:26:07.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92462025-03-04T03:26:07.496ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
92472025-03-04T03:26:07.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92482025-03-04T03:26:07.500ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
92492025-03-04T03:26:07.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92502025-03-04T03:26:07.504ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
92512025-03-04T03:26:07.504ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
92522025-03-04T03:26:07.504ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c663fe82-a04a-4d4f-828c-d0e5b70fea07", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
92532025-03-04T03:26:07.507ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 301.985111ms iteration = 17 status = {}
92542025-03-04T03:26:07.517ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92552025-03-04T03:26:07.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
92562025-03-04T03:26:07.519ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
92572025-03-04T03:26:07.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
92582025-03-04T03:26:07.521ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92592025-03-04T03:26:07.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
92602025-03-04T03:26:07.522ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
92612025-03-04T03:26:07.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
92622025-03-04T03:26:07.528ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 323.471494ms iteration = 17 status = {}
92632025-03-04T03:26:07.534ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
92642025-03-04T03:26:07.541ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
92652025-03-04T03:26:07.541ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 335.746075ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
92662025-03-04T03:26:07.546ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e15ef500-148d-4fd4-a638-254916674f29 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
92672025-03-04T03:26:07.547ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e15ef500-148d-4fd4-a638-254916674f29 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
92682025-03-04T03:26:07.549ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2220 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e15ef500-148d-4fd4-a638-254916674f29 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
92692025-03-04T03:26:07.564ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
92702025-03-04T03:26:07.564ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "941fbb3a-58c2-49f7-bfcf-7a8d87f32067", "content-length": "1", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
92712025-03-04T03:26:07.565ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
92722025-03-04T03:26:07.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
92732025-03-04T03:26:07.568ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 363.554206ms iteration = 17 status = null
92742025-03-04T03:26:07.591ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
92752025-03-04T03:26:07.591ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e1e81435-05d8-40cb-9085-5cfa1ecb096b", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
92762025-03-04T03:26:07.591ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
92772025-03-04T03:26:07.593ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "d33a2856-9289-4119-b722-a8dd9816afa2", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
92782025-03-04T03:26:07.593ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
92792025-03-04T03:26:07.594ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
92802025-03-04T03:26:07.594ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 389.758516ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
92812025-03-04T03:26:07.620ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
92822025-03-04T03:26:07.620ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "56230be4-3a20-435c-b80e-17aaae831551", "content-length": "691", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
92832025-03-04T03:26:07.620ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
92842025-03-04T03:26:07.620ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
92852025-03-04T03:26:07.620ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
92862025-03-04T03:26:07.622ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 417.285774ms iteration = 17 status = {}
92872025-03-04T03:26:07.624ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
92882025-03-04T03:26:07.625ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "abacec69-9f33-40b1-964f-3ea9d8a307ed", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
92892025-03-04T03:26:07.625ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
92902025-03-04T03:26:07.627ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "b4bf3a17-b117-4354-8ef3-24892f736f0b", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
92912025-03-04T03:26:07.627ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
92922025-03-04T03:26:07.628ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "1130faa3-8014-4e66-9c57-b893b546b6ca", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
92932025-03-04T03:26:07.628ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
92942025-03-04T03:26:07.629ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
92952025-03-04T03:26:07.629ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 425.158321ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
92962025-03-04T03:26:07.654ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92972025-03-04T03:26:07.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92982025-03-04T03:26:07.656ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
92992025-03-04T03:26:07.656ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93002025-03-04T03:26:07.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
93012025-03-04T03:26:07.666ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
93022025-03-04T03:26:07.666ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "585efb27-7a29-4de1-a9ee-25a6d00d8bec", "content-length": "134", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
93032025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
93042025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
93052025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
93062025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
93072025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
93082025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
93092025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
93102025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
93112025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
93122025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
93132025-03-04T03:26:07.666ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
93142025-03-04T03:26:07.666ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93152025-03-04T03:26:07.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
93162025-03-04T03:26:07.669ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
93172025-03-04T03:26:07.669ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f7b8a03-6074-4b2a-8111-70ca61d401bb", "content-length": "2", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
93182025-03-04T03:26:07.669ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
93192025-03-04T03:26:07.669ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "3676c3bc-2bf6-4ad8-816c-df253b591a96", "content-length": "51", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
93202025-03-04T03:26:07.669ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
93212025-03-04T03:26:07.669ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
93222025-03-04T03:26:07.766ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "aede83da-9d32-4b6a-8fb6-5efbc3be3ecb", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
93232025-03-04T03:26:07.766ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
93242025-03-04T03:26:07.766ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
93252025-03-04T03:26:07.766ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d0bac55-302c-4366-b018-52abd2ebf9fd", "content-length": "682", "date": "Tue, 04 Mar 2025 03:26:07 GMT"} })
93262025-03-04T03:26:07.766ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93272025-03-04T03:26:07.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
93282025-03-04T03:26:07.769ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93292025-03-04T03:26:07.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
93302025-03-04T03:26:07.773ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93312025-03-04T03:26:07.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
93322025-03-04T03:26:07.776ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93332025-03-04T03:26:07.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
93342025-03-04T03:26:07.783ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 578.978466ms iteration = 17 status = {}
93352025-03-04T03:26:07.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 472 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = bfc31331-448b-4762-a297-752517701522 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
93362025-03-04T03:26:08.569ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a485a7d5-a3e0-433b-a1a3-b7273e248d34 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93372025-03-04T03:26:08.569ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a485a7d5-a3e0-433b-a1a3-b7273e248d34 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93382025-03-04T03:26:08.571ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2487 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a485a7d5-a3e0-433b-a1a3-b7273e248d34 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93392025-03-04T03:26:09.599ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e1a0fa5-7e51-4dea-a445-15e85da6cdae roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93402025-03-04T03:26:09.600ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e1a0fa5-7e51-4dea-a445-15e85da6cdae resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93412025-03-04T03:26:09.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2143 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e1a0fa5-7e51-4dea-a445-15e85da6cdae response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93422025-03-04T03:26:10.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = abc90056-fab5-4987-9eae-a4c7d10b6843 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93432025-03-04T03:26:10.620ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = abc90056-fab5-4987-9eae-a4c7d10b6843 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93442025-03-04T03:26:10.622ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2540 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = abc90056-fab5-4987-9eae-a4c7d10b6843 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93452025-03-04T03:26:11.641ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a00a602f-0848-4eb3-8355-8102e873e5d4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93462025-03-04T03:26:11.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a00a602f-0848-4eb3-8355-8102e873e5d4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93472025-03-04T03:26:11.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2787 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a00a602f-0848-4eb3-8355-8102e873e5d4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93482025-03-04T03:26:12.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08058b21-1284-4f3f-9e0e-309eb0eaffab roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93492025-03-04T03:26:12.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08058b21-1284-4f3f-9e0e-309eb0eaffab resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93502025-03-04T03:26:12.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08058b21-1284-4f3f-9e0e-309eb0eaffab response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93512025-03-04T03:26:13.686ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 78ad58f8-85d2-4057-93b1-050706866eb0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93522025-03-04T03:26:13.687ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 78ad58f8-85d2-4057-93b1-050706866eb0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93532025-03-04T03:26:13.689ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2589 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 78ad58f8-85d2-4057-93b1-050706866eb0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93542025-03-04T03:26:14.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a9954f9-96b5-41a1-af6c-8e0c0ef5bcf3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93552025-03-04T03:26:14.708ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a9954f9-96b5-41a1-af6c-8e0c0ef5bcf3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93562025-03-04T03:26:14.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1540 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a9954f9-96b5-41a1-af6c-8e0c0ef5bcf3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93572025-03-04T03:26:15.729ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = baddc0e7-42d1-4387-9d32-b9615451f7d1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93582025-03-04T03:26:15.729ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = baddc0e7-42d1-4387-9d32-b9615451f7d1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93592025-03-04T03:26:15.731ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2097 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = baddc0e7-42d1-4387-9d32-b9615451f7d1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93602025-03-04T03:26:16.750ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4b18a096-1077-4223-b719-f4cd61b93e6a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93612025-03-04T03:26:16.750ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4b18a096-1077-4223-b719-f4cd61b93e6a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93622025-03-04T03:26:16.752ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2467 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4b18a096-1077-4223-b719-f4cd61b93e6a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93632025-03-04T03:26:17.204ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
93642025-03-04T03:26:17.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93652025-03-04T03:26:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93662025-03-04T03:26:17.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
93672025-03-04T03:26:17.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
93682025-03-04T03:26:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
93692025-03-04T03:26:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93702025-03-04T03:26:17.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
93712025-03-04T03:26:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
93722025-03-04T03:26:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
93732025-03-04T03:26:17.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93742025-03-04T03:26:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93752025-03-04T03:26:17.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93762025-03-04T03:26:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93772025-03-04T03:26:17.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93782025-03-04T03:26:17.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
93792025-03-04T03:26:17.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93802025-03-04T03:26:17.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93812025-03-04T03:26:17.224ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
93822025-03-04T03:26:17.224ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.681411ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
93832025-03-04T03:26:17.240ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.28231ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
93842025-03-04T03:26:17.771ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f23b6d79-255b-4d34-b380-1e7e2d1e318a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93852025-03-04T03:26:17.772ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f23b6d79-255b-4d34-b380-1e7e2d1e318a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93862025-03-04T03:26:17.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2090 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f23b6d79-255b-4d34-b380-1e7e2d1e318a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93872025-03-04T03:26:17.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = b3a34d0e-5928-4736-bf76-420884034e4b roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
93882025-03-04T03:26:17.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = b3a34d0e-5928-4736-bf76-420884034e4b resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
93892025-03-04T03:26:17.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2992 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = b3a34d0e-5928-4736-bf76-420884034e4b response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
93902025-03-04T03:26:17.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 576d63a6-70c5-4389-adbc-fa1a7ff28f64 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
93912025-03-04T03:26:17.894ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 576d63a6-70c5-4389-adbc-fa1a7ff28f64 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
93922025-03-04T03:26:17.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1943 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 576d63a6-70c5-4389-adbc-fa1a7ff28f64 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
93932025-03-04T03:26:17.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 473 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = d728841b-33d0-405d-8e26-40e474d2582d response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
93942025-03-04T03:26:18.793ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c35b1f0c-5aa7-4fc4-8b67-de9e3dfde12b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93952025-03-04T03:26:18.793ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c35b1f0c-5aa7-4fc4-8b67-de9e3dfde12b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93962025-03-04T03:26:18.795ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2040 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c35b1f0c-5aa7-4fc4-8b67-de9e3dfde12b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93972025-03-04T03:26:19.815ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6bcfd30f-4ac0-4f5c-b5ce-3ee86eadec74 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93982025-03-04T03:26:19.815ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6bcfd30f-4ac0-4f5c-b5ce-3ee86eadec74 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
93992025-03-04T03:26:19.817ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2029 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6bcfd30f-4ac0-4f5c-b5ce-3ee86eadec74 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94002025-03-04T03:26:20.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 461b784b-1c2f-4493-99b8-8bd9056cb786 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94012025-03-04T03:26:20.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 461b784b-1c2f-4493-99b8-8bd9056cb786 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94022025-03-04T03:26:20.838ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2283 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 461b784b-1c2f-4493-99b8-8bd9056cb786 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94032025-03-04T03:26:21.858ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2fe3a822-8304-4508-9bbb-588630073ca8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94042025-03-04T03:26:21.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2fe3a822-8304-4508-9bbb-588630073ca8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94052025-03-04T03:26:21.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1830 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2fe3a822-8304-4508-9bbb-588630073ca8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94062025-03-04T03:26:22.879ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef893f1f-1bf0-410f-acd0-2752db2bf945 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94072025-03-04T03:26:22.879ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef893f1f-1bf0-410f-acd0-2752db2bf945 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94082025-03-04T03:26:22.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2592 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef893f1f-1bf0-410f-acd0-2752db2bf945 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94092025-03-04T03:26:23.900ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3230cac-2827-46cd-b365-6ee067b834bd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94102025-03-04T03:26:23.901ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3230cac-2827-46cd-b365-6ee067b834bd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94112025-03-04T03:26:23.903ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2943 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e3230cac-2827-46cd-b365-6ee067b834bd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94122025-03-04T03:26:24.923ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 14181c9c-7bac-4288-a8e4-4b534feb30cb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94132025-03-04T03:26:24.923ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 14181c9c-7bac-4288-a8e4-4b534feb30cb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94142025-03-04T03:26:24.925ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2664 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 14181c9c-7bac-4288-a8e4-4b534feb30cb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94152025-03-04T03:26:25.945ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d207abd-66aa-466f-b7aa-8d7b3f2ce489 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94162025-03-04T03:26:25.945ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d207abd-66aa-466f-b7aa-8d7b3f2ce489 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94172025-03-04T03:26:25.947ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2078 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d207abd-66aa-466f-b7aa-8d7b3f2ce489 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94182025-03-04T03:26:26.969ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e5743b98-8706-4974-a965-2464bb0b20b5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94192025-03-04T03:26:26.969ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e5743b98-8706-4974-a965-2464bb0b20b5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94202025-03-04T03:26:26.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1728 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e5743b98-8706-4974-a965-2464bb0b20b5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94212025-03-04T03:26:27.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
94222025-03-04T03:26:27.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94232025-03-04T03:26:27.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94242025-03-04T03:26:27.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
94252025-03-04T03:26:27.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94262025-03-04T03:26:27.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94272025-03-04T03:26:27.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94282025-03-04T03:26:27.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94292025-03-04T03:26:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94302025-03-04T03:26:27.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.178491ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
94312025-03-04T03:26:27.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
94322025-03-04T03:26:27.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94332025-03-04T03:26:27.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94342025-03-04T03:26:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94352025-03-04T03:26:27.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94362025-03-04T03:26:27.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94372025-03-04T03:26:27.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
94382025-03-04T03:26:27.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94392025-03-04T03:26:27.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94402025-03-04T03:26:27.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
94412025-03-04T03:26:27.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.901847ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
94422025-03-04T03:26:27.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 484 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 8f17953f-ab4f-4e9a-a9c3-9a94287bb56f response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
94432025-03-04T03:26:27.990ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 90062b84-8edf-4fe8-b559-8ab98ae45867 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94442025-03-04T03:26:27.990ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 90062b84-8edf-4fe8-b559-8ab98ae45867 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94452025-03-04T03:26:27.992ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2130 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 90062b84-8edf-4fe8-b559-8ab98ae45867 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94462025-03-04T03:26:29.011ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab3f7859-602a-40ca-b4ff-5ad997ee6fac roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94472025-03-04T03:26:29.012ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab3f7859-602a-40ca-b4ff-5ad997ee6fac resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94482025-03-04T03:26:29.014ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2202 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab3f7859-602a-40ca-b4ff-5ad997ee6fac response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94492025-03-04T03:26:30.033ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8e51715-ee1b-465c-91ab-448c88f0d4a5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94502025-03-04T03:26:30.033ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8e51715-ee1b-465c-91ab-448c88f0d4a5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94512025-03-04T03:26:30.035ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1989 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8e51715-ee1b-465c-91ab-448c88f0d4a5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94522025-03-04T03:26:31.054ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb5bffbd-5e69-48bf-bd88-bd3396954703 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94532025-03-04T03:26:31.055ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb5bffbd-5e69-48bf-bd88-bd3396954703 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94542025-03-04T03:26:31.056ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1650 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb5bffbd-5e69-48bf-bd88-bd3396954703 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94552025-03-04T03:26:32.076ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 805cb03d-fa56-45de-9195-a2b821de146f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94562025-03-04T03:26:32.076ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 805cb03d-fa56-45de-9195-a2b821de146f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94572025-03-04T03:26:32.078ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2590 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 805cb03d-fa56-45de-9195-a2b821de146f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94582025-03-04T03:26:33.098ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9feebfdb-0013-4eb2-96c1-36a5da4ba212 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94592025-03-04T03:26:33.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9feebfdb-0013-4eb2-96c1-36a5da4ba212 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94602025-03-04T03:26:33.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2274 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9feebfdb-0013-4eb2-96c1-36a5da4ba212 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94612025-03-04T03:26:34.119ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68b01e1d-15c6-47d1-8aa4-a1083f1191eb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94622025-03-04T03:26:34.120ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68b01e1d-15c6-47d1-8aa4-a1083f1191eb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94632025-03-04T03:26:34.122ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2643 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 68b01e1d-15c6-47d1-8aa4-a1083f1191eb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94642025-03-04T03:26:35.141ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7fe624d8-254c-4747-8cdf-ea24da765d32 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94652025-03-04T03:26:35.141ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7fe624d8-254c-4747-8cdf-ea24da765d32 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94662025-03-04T03:26:35.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2768 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7fe624d8-254c-4747-8cdf-ea24da765d32 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94672025-03-04T03:26:36.163ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca6bd59a-cdf9-4f10-8fb4-89e496e20dfa roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94682025-03-04T03:26:36.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca6bd59a-cdf9-4f10-8fb4-89e496e20dfa resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94692025-03-04T03:26:36.165ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1897 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca6bd59a-cdf9-4f10-8fb4-89e496e20dfa response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94702025-03-04T03:26:37.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4722d0cc-3ef1-46fc-b1a2-b92299d14204 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94712025-03-04T03:26:37.184ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4722d0cc-3ef1-46fc-b1a2-b92299d14204 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94722025-03-04T03:26:37.186ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2455 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4722d0cc-3ef1-46fc-b1a2-b92299d14204 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
94732025-03-04T03:26:37.204ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
94742025-03-04T03:26:37.204ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
94752025-03-04T03:26:37.204ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
94762025-03-04T03:26:37.204ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
94772025-03-04T03:26:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
94782025-03-04T03:26:37.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
94792025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
94802025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
94812025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
94822025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
94832025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
94842025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
94852025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94862025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
94872025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
94882025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
94892025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
94902025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
94912025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94922025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
94932025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
94942025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
94952025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94962025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94972025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
94982025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
94992025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
95002025-03-04T03:26:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
95012025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
95022025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95032025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
95042025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
95052025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
95062025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
95072025-03-04T03:26:37.206ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
95082025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
95092025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
95102025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95112025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
95122025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
95132025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95142025-03-04T03:26:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95152025-03-04T03:26:37.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1509, tv_nsec: 402627493 } })
95162025-03-04T03:26:37.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
95172025-03-04T03:26:37.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1509, tv_nsec: 403010068 } })
95182025-03-04T03:26:37.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
95192025-03-04T03:26:37.207ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95202025-03-04T03:26:37.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95212025-03-04T03:26:37.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95222025-03-04T03:26:37.209ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.972119ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
95232025-03-04T03:26:37.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95242025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
95252025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
95262025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
95272025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
95282025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
95292025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
95302025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
95312025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
95322025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
95332025-03-04T03:26:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
95342025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
95352025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
95362025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
95372025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
95382025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
95392025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
95402025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
95412025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
95422025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95432025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
95442025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95452025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
95462025-03-04T03:26:37.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95472025-03-04T03:26:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
95482025-03-04T03:26:37.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
95492025-03-04T03:26:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95502025-03-04T03:26:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
95512025-03-04T03:26:37.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95522025-03-04T03:26:37.214ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95532025-03-04T03:26:37.214ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95542025-03-04T03:26:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95552025-03-04T03:26:37.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95562025-03-04T03:26:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
95572025-03-04T03:26:37.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
95582025-03-04T03:26:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
95592025-03-04T03:26:37.215ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
95602025-03-04T03:26:37.215ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
95612025-03-04T03:26:37.215ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
95622025-03-04T03:26:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
95632025-03-04T03:26:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
95642025-03-04T03:26:37.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
95652025-03-04T03:26:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
95662025-03-04T03:26:37.217ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
95672025-03-04T03:26:37.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95682025-03-04T03:26:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95692025-03-04T03:26:37.218ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
95702025-03-04T03:26:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
95712025-03-04T03:26:37.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95722025-03-04T03:26:37.220ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
95732025-03-04T03:26:37.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95742025-03-04T03:26:37.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
95752025-03-04T03:26:37.221ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
95762025-03-04T03:26:37.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
95772025-03-04T03:26:37.221ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1509, tv_nsec: 417281023 } })
95782025-03-04T03:26:37.221ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
95792025-03-04T03:26:37.222ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
95802025-03-04T03:26:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
95812025-03-04T03:26:37.223ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95822025-03-04T03:26:37.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95832025-03-04T03:26:37.226ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
95842025-03-04T03:26:37.226ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95852025-03-04T03:26:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95862025-03-04T03:26:37.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
95872025-03-04T03:26:37.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95882025-03-04T03:26:37.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95892025-03-04T03:26:37.233ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
95902025-03-04T03:26:37.233ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 26.960441ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
95912025-03-04T03:26:37.249ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
95922025-03-04T03:26:37.249ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 43.856948ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
95932025-03-04T03:26:37.255ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 44.746528ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
95942025-03-04T03:26:37.260ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
95952025-03-04T03:26:37.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
95962025-03-04T03:26:37.261ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
95972025-03-04T03:26:37.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
95982025-03-04T03:26:37.263ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 52.516884ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
95992025-03-04T03:26:37.265ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 54.754481ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
96002025-03-04T03:26:37.266ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
96012025-03-04T03:26:37.266ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
96022025-03-04T03:26:37.266ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
96032025-03-04T03:26:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
96042025-03-04T03:26:37.267ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
96052025-03-04T03:26:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
96062025-03-04T03:26:37.269ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.923341ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
96072025-03-04T03:26:37.271ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
96082025-03-04T03:26:37.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 61.282091ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
96092025-03-04T03:26:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
96102025-03-04T03:26:37.278ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96112025-03-04T03:26:37.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
96122025-03-04T03:26:37.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
96132025-03-04T03:26:37.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
96142025-03-04T03:26:37.304ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
96152025-03-04T03:26:37.304ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
96162025-03-04T03:26:37.305ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a9fb106-8e0e-49d2-9118-9398187adab7", "content-length": "26", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96172025-03-04T03:26:37.305ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
96182025-03-04T03:26:37.307ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
96192025-03-04T03:26:37.317ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
96202025-03-04T03:26:37.317ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
96212025-03-04T03:26:37.318ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "130029d5-91d0-471f-8134-00e6ea88ff3d", "content-length": "26", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96222025-03-04T03:26:37.318ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
96232025-03-04T03:26:37.319ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
96242025-03-04T03:26:37.334ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96252025-03-04T03:26:37.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96262025-03-04T03:26:37.346ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
96272025-03-04T03:26:37.346ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
96282025-03-04T03:26:37.346ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8c4af56-d170-49fa-ab39-0c2f37bbc47f", "content-length": "372", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96292025-03-04T03:26:37.346ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96302025-03-04T03:26:37.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96312025-03-04T03:26:37.348ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96322025-03-04T03:26:37.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96332025-03-04T03:26:37.349ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96342025-03-04T03:26:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
96352025-03-04T03:26:37.351ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
96362025-03-04T03:26:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
96372025-03-04T03:26:37.360ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
96382025-03-04T03:26:37.360ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
96392025-03-04T03:26:37.360ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2469143c-86f5-4a8f-8237-72e6cced43b3", "content-length": "26", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96402025-03-04T03:26:37.360ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
96412025-03-04T03:26:37.362ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
96422025-03-04T03:26:37.384ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
96432025-03-04T03:26:37.384ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
96442025-03-04T03:26:37.384ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 178.382676ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
96452025-03-04T03:26:37.392ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
96462025-03-04T03:26:37.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96472025-03-04T03:26:37.394ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96482025-03-04T03:26:37.395ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
96492025-03-04T03:26:37.395ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96502025-03-04T03:26:37.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96512025-03-04T03:26:37.396ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 190.911479ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
96522025-03-04T03:26:37.396ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96532025-03-04T03:26:37.398ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
96542025-03-04T03:26:37.398ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 193.257109ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
96552025-03-04T03:26:37.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
96562025-03-04T03:26:37.400ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96572025-03-04T03:26:37.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96582025-03-04T03:26:37.407ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 202.057807ms iteration = 18 status = 0
96592025-03-04T03:26:37.417ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
96602025-03-04T03:26:37.417ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
96612025-03-04T03:26:37.417ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a10033ec-4fec-4056-abdb-2e603a98e1e9", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96622025-03-04T03:26:37.417ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96632025-03-04T03:26:37.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96642025-03-04T03:26:37.420ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96652025-03-04T03:26:37.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96662025-03-04T03:26:37.423ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96672025-03-04T03:26:37.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96682025-03-04T03:26:37.425ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 215.029375ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
96692025-03-04T03:26:37.426ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
96702025-03-04T03:26:37.426ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
96712025-03-04T03:26:37.426ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "653cd284-dc3e-42db-a9c7-ff64410835af", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96722025-03-04T03:26:37.427ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 221.427954ms iteration = 18 status = {}
96732025-03-04T03:26:37.434ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96742025-03-04T03:26:37.436ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
96752025-03-04T03:26:37.436ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96762025-03-04T03:26:37.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96772025-03-04T03:26:37.445ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 240.588648ms iteration = 18 status = {}
96782025-03-04T03:26:37.461ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 256.527723ms iteration = 20 status = {"physical_disks_added":0}
96792025-03-04T03:26:37.477ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
96802025-03-04T03:26:37.478ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "f65168f9-14c4-4206-9ca2-546111542ee8", "content-length": "1", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96812025-03-04T03:26:37.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
96822025-03-04T03:26:37.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
96832025-03-04T03:26:37.481ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 276.859343ms iteration = 18 status = null
96842025-03-04T03:26:37.556ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
96852025-03-04T03:26:37.556ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "12fc0776-6422-403b-9e86-13234585b32b", "content-length": "691", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96862025-03-04T03:26:37.556ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
96872025-03-04T03:26:37.556ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
96882025-03-04T03:26:37.556ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
96892025-03-04T03:26:37.557ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 352.594651ms iteration = 18 status = {}
96902025-03-04T03:26:37.577ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96912025-03-04T03:26:37.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96922025-03-04T03:26:37.579ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
96932025-03-04T03:26:37.579ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96942025-03-04T03:26:37.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96952025-03-04T03:26:37.589ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
96962025-03-04T03:26:37.589ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ff9e895-2448-47b2-9eb7-1efe4cb462c6", "content-length": "134", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
96972025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
96982025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
96992025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
97002025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
97012025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
97022025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
97032025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
97042025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
97052025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
97062025-03-04T03:26:37.589ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
97072025-03-04T03:26:37.590ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
97082025-03-04T03:26:37.590ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97092025-03-04T03:26:37.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97102025-03-04T03:26:37.592ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
97112025-03-04T03:26:37.592ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c18eae0-4e1c-4267-8db2-1548a8301f77", "content-length": "2", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
97122025-03-04T03:26:37.592ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
97132025-03-04T03:26:37.592ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad2c9fa2-21d3-4bf4-8c9e-7afb67cad252", "content-length": "51", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
97142025-03-04T03:26:37.592ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
97152025-03-04T03:26:37.592ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
97162025-03-04T03:26:37.692ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "47e3392d-d6bf-4fb6-90ff-59651753540c", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
97172025-03-04T03:26:37.692ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
97182025-03-04T03:26:37.692ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
97192025-03-04T03:26:37.692ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "76677708-6b40-444f-80ba-6de979bd6f6a", "content-length": "682", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
97202025-03-04T03:26:37.692ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97212025-03-04T03:26:37.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97222025-03-04T03:26:37.696ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97232025-03-04T03:26:37.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97242025-03-04T03:26:37.700ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97252025-03-04T03:26:37.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97262025-03-04T03:26:37.703ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97272025-03-04T03:26:37.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97282025-03-04T03:26:37.711ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 505.706317ms iteration = 18 status = {}
97292025-03-04T03:26:37.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 495 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 625862cd-1235-41e3-a6da-ee9d3185c1bd response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
97302025-03-04T03:26:37.915ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
97312025-03-04T03:26:37.915ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
97322025-03-04T03:26:37.945ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
97332025-03-04T03:26:37.945ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:60427
97342025-03-04T03:26:37.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:60427 req_id = 86162c90-7d89-44c9-90d4-6184a0403fa6 roles = RoleSet { roles: {} } uri = /metrics/producers
97352025-03-04T03:26:37.946ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:60427 req_id = 86162c90-7d89-44c9-90d4-6184a0403fa6 resource = Database result = Ok(()) uri = /metrics/producers
97362025-03-04T03:26:37.981ZINFOnexus: assigned collector to new producer collector_id = ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a file = nexus/src/app/oximeter.rs:76 producer_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
97372025-03-04T03:26:37.981ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 35711 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:60427 req_id = 86162c90-7d89-44c9-90d4-6184a0403fa6 response_code = 201 uri = /metrics/producers
97382025-03-04T03:26:37.981ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "86162c90-7d89-44c9-90d4-6184a0403fa6", "content-length": "41", "date": "Tue, 04 Mar 2025 03:26:37 GMT"} })
97392025-03-04T03:26:37.983ZDEBGnexus (producer-registration-task): registered with nexus successfully
97402025-03-04T03:26:37.983ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
97412025-03-04T03:26:38.205ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a4d56fce-a9fa-47f6-a2bb-64b169180817 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97422025-03-04T03:26:38.205ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a4d56fce-a9fa-47f6-a2bb-64b169180817 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97432025-03-04T03:26:38.207ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2224 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a4d56fce-a9fa-47f6-a2bb-64b169180817 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97442025-03-04T03:26:39.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f461335-ef49-4b21-a119-a3ffc0797860 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97452025-03-04T03:26:39.227ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f461335-ef49-4b21-a119-a3ffc0797860 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97462025-03-04T03:26:39.229ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2480 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f461335-ef49-4b21-a119-a3ffc0797860 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97472025-03-04T03:26:40.246ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8e6796c-c550-408c-adc5-1c481cc7e45e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97482025-03-04T03:26:40.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8e6796c-c550-408c-adc5-1c481cc7e45e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97492025-03-04T03:26:40.248ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2264 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8e6796c-c550-408c-adc5-1c481cc7e45e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97502025-03-04T03:26:41.267ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4639f66f-9eef-4473-bada-fa461bd97b95 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97512025-03-04T03:26:41.268ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4639f66f-9eef-4473-bada-fa461bd97b95 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97522025-03-04T03:26:41.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2516 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4639f66f-9eef-4473-bada-fa461bd97b95 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97532025-03-04T03:26:42.288ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 510c2e01-3b4c-4e1d-aff6-4f4d05875814 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97542025-03-04T03:26:42.289ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 510c2e01-3b4c-4e1d-aff6-4f4d05875814 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97552025-03-04T03:26:42.291ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2770 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 510c2e01-3b4c-4e1d-aff6-4f4d05875814 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97562025-03-04T03:26:43.310ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bfb551e6-6604-4b22-8577-d31c470094e1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97572025-03-04T03:26:43.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bfb551e6-6604-4b22-8577-d31c470094e1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97582025-03-04T03:26:43.313ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2301 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bfb551e6-6604-4b22-8577-d31c470094e1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97592025-03-04T03:26:44.333ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0c79f327-f667-49c0-af73-abfd0199f860 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97602025-03-04T03:26:44.334ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0c79f327-f667-49c0-af73-abfd0199f860 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97612025-03-04T03:26:44.335ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1949 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0c79f327-f667-49c0-af73-abfd0199f860 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97622025-03-04T03:26:45.354ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93cec54a-c09c-4113-951b-bcd126d84f64 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97632025-03-04T03:26:45.354ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93cec54a-c09c-4113-951b-bcd126d84f64 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97642025-03-04T03:26:45.356ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2080 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93cec54a-c09c-4113-951b-bcd126d84f64 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97652025-03-04T03:26:46.375ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2090597b-f31b-48dc-8cec-e95e6b0c880e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97662025-03-04T03:26:46.375ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2090597b-f31b-48dc-8cec-e95e6b0c880e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97672025-03-04T03:26:46.377ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2423 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2090597b-f31b-48dc-8cec-e95e6b0c880e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97682025-03-04T03:26:47.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
97692025-03-04T03:26:47.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97702025-03-04T03:26:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97712025-03-04T03:26:47.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
97722025-03-04T03:26:47.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
97732025-03-04T03:26:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
97742025-03-04T03:26:47.234ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
97752025-03-04T03:26:47.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97762025-03-04T03:26:47.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
97772025-03-04T03:26:47.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
97782025-03-04T03:26:47.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97792025-03-04T03:26:47.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97802025-03-04T03:26:47.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97812025-03-04T03:26:47.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97822025-03-04T03:26:47.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97832025-03-04T03:26:47.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
97842025-03-04T03:26:47.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97852025-03-04T03:26:47.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97862025-03-04T03:26:47.255ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
97872025-03-04T03:26:47.255ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.882964ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
97882025-03-04T03:26:47.265ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.702212ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
97892025-03-04T03:26:47.397ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 91020c84-8787-4800-abc9-9356b8866cd9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97902025-03-04T03:26:47.397ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 91020c84-8787-4800-abc9-9356b8866cd9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97912025-03-04T03:26:47.399ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1796 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 91020c84-8787-4800-abc9-9356b8866cd9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
97922025-03-04T03:26:47.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 71b4cb1c-967c-4992-aea3-514b9a0324d1 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
97932025-03-04T03:26:47.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 71b4cb1c-967c-4992-aea3-514b9a0324d1 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
97942025-03-04T03:26:47.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3052 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = 71b4cb1c-967c-4992-aea3-514b9a0324d1 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
97952025-03-04T03:26:47.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = be999116-f94a-4650-8c7d-1ec545d8acf5 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
97962025-03-04T03:26:47.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = be999116-f94a-4650-8c7d-1ec545d8acf5 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
97972025-03-04T03:26:47.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2079 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44097 req_id = be999116-f94a-4650-8c7d-1ec545d8acf5 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
97982025-03-04T03:26:47.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 464 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 0c4cf146-c491-4fe2-9600-438e1b1e074f response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
97992025-03-04T03:26:48.418ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a66344ab-5569-4623-9899-f7b06d8718c2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98002025-03-04T03:26:48.419ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a66344ab-5569-4623-9899-f7b06d8718c2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98012025-03-04T03:26:48.420ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1963 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a66344ab-5569-4623-9899-f7b06d8718c2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98022025-03-04T03:26:49.439ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61606814-3d34-4335-855d-e556176aa227 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98032025-03-04T03:26:49.439ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61606814-3d34-4335-855d-e556176aa227 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98042025-03-04T03:26:49.441ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1982 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61606814-3d34-4335-855d-e556176aa227 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98052025-03-04T03:26:50.461ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 72582a02-6a5e-4399-8a84-756e49a63426 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98062025-03-04T03:26:50.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 72582a02-6a5e-4399-8a84-756e49a63426 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98072025-03-04T03:26:50.463ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2268 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 72582a02-6a5e-4399-8a84-756e49a63426 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98082025-03-04T03:26:51.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 81b93bcd-df89-41f7-84a9-3004a0430b06 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98092025-03-04T03:26:51.482ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 81b93bcd-df89-41f7-84a9-3004a0430b06 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98102025-03-04T03:26:51.483ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1577 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 81b93bcd-df89-41f7-84a9-3004a0430b06 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98112025-03-04T03:26:52.502ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ee61149c-d2b7-47b3-bdd5-604dbda7812e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98122025-03-04T03:26:52.502ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ee61149c-d2b7-47b3-bdd5-604dbda7812e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98132025-03-04T03:26:52.504ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2393 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ee61149c-d2b7-47b3-bdd5-604dbda7812e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98142025-03-04T03:26:53.524ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d20fd3f1-725e-4da5-9a7e-35ceb6007479 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98152025-03-04T03:26:53.524ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d20fd3f1-725e-4da5-9a7e-35ceb6007479 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98162025-03-04T03:26:53.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2201 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d20fd3f1-725e-4da5-9a7e-35ceb6007479 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98172025-03-04T03:26:54.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 388a937b-5080-452c-88bf-ee538efd302d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98182025-03-04T03:26:54.545ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 388a937b-5080-452c-88bf-ee538efd302d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98192025-03-04T03:26:54.547ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2212 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 388a937b-5080-452c-88bf-ee538efd302d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98202025-03-04T03:26:55.565ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dfa87ec7-20d5-41ae-8481-74f469c37326 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98212025-03-04T03:26:55.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dfa87ec7-20d5-41ae-8481-74f469c37326 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98222025-03-04T03:26:55.568ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2395 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dfa87ec7-20d5-41ae-8481-74f469c37326 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98232025-03-04T03:26:56.587ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 013202c1-a5ab-4bbe-9a40-6aadeba41807 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98242025-03-04T03:26:56.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 013202c1-a5ab-4bbe-9a40-6aadeba41807 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98252025-03-04T03:26:56.588ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1695 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 013202c1-a5ab-4bbe-9a40-6aadeba41807 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98262025-03-04T03:26:57.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
98272025-03-04T03:26:57.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98282025-03-04T03:26:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98292025-03-04T03:26:57.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
98302025-03-04T03:26:57.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98312025-03-04T03:26:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98322025-03-04T03:26:57.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98332025-03-04T03:26:57.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98342025-03-04T03:26:57.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98352025-03-04T03:26:57.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
98362025-03-04T03:26:57.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98372025-03-04T03:26:57.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98382025-03-04T03:26:57.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98392025-03-04T03:26:57.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98402025-03-04T03:26:57.268ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.717079ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
98412025-03-04T03:26:57.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98422025-03-04T03:26:57.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
98432025-03-04T03:26:57.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98442025-03-04T03:26:57.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98452025-03-04T03:26:57.290ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
98462025-03-04T03:26:57.290ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 84.503036ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
98472025-03-04T03:26:57.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 44da446e-f33c-475c-946e-7ec050d4a331 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98482025-03-04T03:26:57.608ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 44da446e-f33c-475c-946e-7ec050d4a331 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98492025-03-04T03:26:57.609ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2107 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 44da446e-f33c-475c-946e-7ec050d4a331 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98502025-03-04T03:26:57.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 537 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = c1057cdd-0bb0-4f7f-ae62-1f280b003b07 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
98512025-03-04T03:26:58.629ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a05eb4d6-c50b-4965-ada1-5586c5b0150e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98522025-03-04T03:26:58.630ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a05eb4d6-c50b-4965-ada1-5586c5b0150e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98532025-03-04T03:26:58.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2450 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a05eb4d6-c50b-4965-ada1-5586c5b0150e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98542025-03-04T03:26:59.652ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d10fcddb-fd4e-430b-b7bd-eb1c2ecfcd98 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98552025-03-04T03:26:59.652ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d10fcddb-fd4e-430b-b7bd-eb1c2ecfcd98 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98562025-03-04T03:26:59.654ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2491 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d10fcddb-fd4e-430b-b7bd-eb1c2ecfcd98 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98572025-03-04T03:27:00.674ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c3ff18d-3b5b-42de-abac-e5ded3ab319a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98582025-03-04T03:27:00.674ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c3ff18d-3b5b-42de-abac-e5ded3ab319a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98592025-03-04T03:27:00.676ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2013 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9c3ff18d-3b5b-42de-abac-e5ded3ab319a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98602025-03-04T03:27:01.695ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0fe4fed2-cec6-4a68-a4bb-201b97481b70 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98612025-03-04T03:27:01.696ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0fe4fed2-cec6-4a68-a4bb-201b97481b70 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98622025-03-04T03:27:01.697ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1973 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0fe4fed2-cec6-4a68-a4bb-201b97481b70 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98632025-03-04T03:27:02.718ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32601025-4e5c-4c9d-a025-8860232727e9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98642025-03-04T03:27:02.718ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32601025-4e5c-4c9d-a025-8860232727e9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98652025-03-04T03:27:02.719ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1809 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 32601025-4e5c-4c9d-a025-8860232727e9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98662025-03-04T03:27:03.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 46f9c863-8e6c-4b7d-a080-534b34696add roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98672025-03-04T03:27:03.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 46f9c863-8e6c-4b7d-a080-534b34696add resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98682025-03-04T03:27:03.741ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2551 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 46f9c863-8e6c-4b7d-a080-534b34696add response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98692025-03-04T03:27:04.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b4eac3a-173c-45f6-b137-d0305af88f6d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98702025-03-04T03:27:04.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b4eac3a-173c-45f6-b137-d0305af88f6d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98712025-03-04T03:27:04.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2177 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b4eac3a-173c-45f6-b137-d0305af88f6d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98722025-03-04T03:27:05.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d3365d3c-436b-4098-8299-2370cfe36ed7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98732025-03-04T03:27:05.782ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d3365d3c-436b-4098-8299-2370cfe36ed7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98742025-03-04T03:27:05.783ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1632 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d3365d3c-436b-4098-8299-2370cfe36ed7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98752025-03-04T03:27:06.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8815289-7124-4ca9-8380-796f3287134c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98762025-03-04T03:27:06.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8815289-7124-4ca9-8380-796f3287134c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98772025-03-04T03:27:06.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2088 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b8815289-7124-4ca9-8380-796f3287134c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
98782025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
98792025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
98802025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
98812025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
98822025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
98832025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
98842025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
98852025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
98862025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
98872025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
98882025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
98892025-03-04T03:27:07.204ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:17:07.204476687 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
98902025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
98912025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
98922025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
98932025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
98942025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
98952025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
98962025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
98972025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
98982025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
98992025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
99002025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
99012025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
99022025-03-04T03:27:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
99032025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
99042025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
99052025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
99062025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99072025-03-04T03:27:07.205ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
99082025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
99092025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
99102025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
99112025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
99122025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
99132025-03-04T03:27:07.205ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
99142025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
99152025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.35\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
99162025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
99172025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
99182025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
99192025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
99202025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
99212025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
99222025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99232025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
99242025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
99252025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
99262025-03-04T03:27:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1539, tv_nsec: 400754652 } })
99272025-03-04T03:27:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1539, tv_nsec: 400752162 } })
99282025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
99292025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
99302025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
99312025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
99322025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
99332025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99342025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
99352025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99362025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
99372025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
99382025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
99392025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.311156ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
99402025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
99412025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
99422025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
99432025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.210575ms iteration = 10 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
99442025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
99452025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
99462025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
99472025-03-04T03:27:07.205ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
99482025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99492025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
99502025-03-04T03:27:07.205ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
99512025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
99522025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
99532025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
99542025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
99552025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
99562025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
99572025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
99582025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
99592025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
99602025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99612025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
99622025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
99632025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99642025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
99652025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
99662025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99672025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
99682025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
99692025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
99702025-03-04T03:27:07.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99712025-03-04T03:27:07.208ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
99722025-03-04T03:27:07.208ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
99732025-03-04T03:27:07.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
99742025-03-04T03:27:07.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
99752025-03-04T03:27:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
99762025-03-04T03:27:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
99772025-03-04T03:27:07.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1539, tv_nsec: 404080403 } })
99782025-03-04T03:27:07.208ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:17:07.204476687 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
99792025-03-04T03:27:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.131881ms iteration = 10 status = {"expiration":"2025-03-04T03:17:07.204476687Z","pruned":[]}
99802025-03-04T03:27:07.208ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
99812025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
99822025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
99832025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
99842025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
99852025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
99862025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
99872025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
99882025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
99892025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
99902025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
99912025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
99922025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
99932025-03-04T03:27:07.210ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1539, tv_nsec: 406420111 } })
99942025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
99952025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
99962025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
99972025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
99982025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
99992025-03-04T03:27:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
100002025-03-04T03:27:07.211ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
100012025-03-04T03:27:07.212ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100022025-03-04T03:27:07.212ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.630833ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
100032025-03-04T03:27:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100042025-03-04T03:27:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
100052025-03-04T03:27:07.214ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
100062025-03-04T03:27:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
100072025-03-04T03:27:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
100082025-03-04T03:27:07.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100092025-03-04T03:27:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100102025-03-04T03:27:07.218ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100112025-03-04T03:27:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100122025-03-04T03:27:07.219ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100132025-03-04T03:27:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100142025-03-04T03:27:07.220ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100152025-03-04T03:27:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100162025-03-04T03:27:07.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100172025-03-04T03:27:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100182025-03-04T03:27:07.223ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100192025-03-04T03:27:07.223ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100202025-03-04T03:27:07.226ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
100212025-03-04T03:27:07.226ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100222025-03-04T03:27:07.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100232025-03-04T03:27:07.227ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
100242025-03-04T03:27:07.234ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100252025-03-04T03:27:07.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
100262025-03-04T03:27:07.249ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100272025-03-04T03:27:07.250ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100282025-03-04T03:27:07.251ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100292025-03-04T03:27:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
100302025-03-04T03:27:07.253ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
100312025-03-04T03:27:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
100322025-03-04T03:27:07.253ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
100332025-03-04T03:27:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
100342025-03-04T03:27:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
100352025-03-04T03:27:07.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
100362025-03-04T03:27:07.255ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
100372025-03-04T03:27:07.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
100382025-03-04T03:27:07.257ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 46.981095ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
100392025-03-04T03:27:07.259ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100402025-03-04T03:27:07.260ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100412025-03-04T03:27:07.261ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
100422025-03-04T03:27:07.261ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
100432025-03-04T03:27:07.261ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
100442025-03-04T03:27:07.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
100452025-03-04T03:27:07.261ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
100462025-03-04T03:27:07.262ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100472025-03-04T03:27:07.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
100482025-03-04T03:27:07.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100492025-03-04T03:27:07.263ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100502025-03-04T03:27:07.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100512025-03-04T03:27:07.263ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100522025-03-04T03:27:07.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
100532025-03-04T03:27:07.263ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
100542025-03-04T03:27:07.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
100552025-03-04T03:27:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
100562025-03-04T03:27:07.264ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
100572025-03-04T03:27:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
100582025-03-04T03:27:07.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100592025-03-04T03:27:07.265ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100602025-03-04T03:27:07.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100612025-03-04T03:27:07.271ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100622025-03-04T03:27:07.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100632025-03-04T03:27:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100642025-03-04T03:27:07.274ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100652025-03-04T03:27:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
100662025-03-04T03:27:07.276ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100672025-03-04T03:27:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100682025-03-04T03:27:07.278ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100692025-03-04T03:27:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100702025-03-04T03:27:07.279ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
100712025-03-04T03:27:07.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100722025-03-04T03:27:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100732025-03-04T03:27:07.280ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100742025-03-04T03:27:07.281ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
100752025-03-04T03:27:07.281ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
100762025-03-04T03:27:07.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
100772025-03-04T03:27:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
100782025-03-04T03:27:07.282ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
100792025-03-04T03:27:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
100802025-03-04T03:27:07.282ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
100812025-03-04T03:27:07.282ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
100822025-03-04T03:27:07.282ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
100832025-03-04T03:27:07.282ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
100842025-03-04T03:27:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
100852025-03-04T03:27:07.282ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100862025-03-04T03:27:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
100872025-03-04T03:27:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
100882025-03-04T03:27:07.283ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1539, tv_nsec: 478552226 } })
100892025-03-04T03:27:07.283ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
100902025-03-04T03:27:07.283ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
100912025-03-04T03:27:07.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 72.830582ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
100922025-03-04T03:27:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
100932025-03-04T03:27:07.283ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100942025-03-04T03:27:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100952025-03-04T03:27:07.285ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
100962025-03-04T03:27:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
100972025-03-04T03:27:07.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100982025-03-04T03:27:07.286ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 81.745782ms iteration = 5 status = {"nsuccess":5}
100992025-03-04T03:27:07.287ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
101002025-03-04T03:27:07.287ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
101012025-03-04T03:27:07.287ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 79.901359ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
101022025-03-04T03:27:07.288ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
101032025-03-04T03:27:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
101042025-03-04T03:27:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
101052025-03-04T03:27:07.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101062025-03-04T03:27:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101072025-03-04T03:27:07.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101082025-03-04T03:27:07.292ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 86.804524ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
101092025-03-04T03:27:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
101102025-03-04T03:27:07.294ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
101112025-03-04T03:27:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
101122025-03-04T03:27:07.303ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 92.605975ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
101132025-03-04T03:27:07.303ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 93.348334ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
101142025-03-04T03:27:07.305ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
101152025-03-04T03:27:07.305ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
101162025-03-04T03:27:07.305ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 100.731284ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
101172025-03-04T03:27:07.312ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
101182025-03-04T03:27:07.312ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 107.489557ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
101192025-03-04T03:27:07.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101202025-03-04T03:27:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101212025-03-04T03:27:07.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101222025-03-04T03:27:07.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101232025-03-04T03:27:07.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101242025-03-04T03:27:07.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
101252025-03-04T03:27:07.322ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101262025-03-04T03:27:07.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101272025-03-04T03:27:07.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
101282025-03-04T03:27:07.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
101292025-03-04T03:27:07.323ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101302025-03-04T03:27:07.323ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101312025-03-04T03:27:07.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
101322025-03-04T03:27:07.324ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.115049ms iteration = 19 status = 0
101332025-03-04T03:27:07.325ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101342025-03-04T03:27:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
101352025-03-04T03:27:07.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101362025-03-04T03:27:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101372025-03-04T03:27:07.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
101382025-03-04T03:27:07.327ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101392025-03-04T03:27:07.327ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
101402025-03-04T03:27:07.327ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
101412025-03-04T03:27:07.327ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 121.795902ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
101422025-03-04T03:27:07.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
101432025-03-04T03:27:07.328ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
101442025-03-04T03:27:07.328ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 122.43647ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
101452025-03-04T03:27:07.328ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
101462025-03-04T03:27:07.328ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
101472025-03-04T03:27:07.328ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
101482025-03-04T03:27:07.328ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101492025-03-04T03:27:07.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101502025-03-04T03:27:07.329ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
101512025-03-04T03:27:07.329ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
101522025-03-04T03:27:07.329ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
101532025-03-04T03:27:07.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
101542025-03-04T03:27:07.329ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101552025-03-04T03:27:07.330ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
101562025-03-04T03:27:07.330ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
101572025-03-04T03:27:07.331ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "40f8566e-653c-4a70-956f-c5a29e6c423e", "content-length": "26", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
101582025-03-04T03:27:07.331ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
101592025-03-04T03:27:07.331ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101602025-03-04T03:27:07.331ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101612025-03-04T03:27:07.331ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 126.49274ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
101622025-03-04T03:27:07.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101632025-03-04T03:27:07.332ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
101642025-03-04T03:27:07.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
101652025-03-04T03:27:07.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
101662025-03-04T03:27:07.333ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101672025-03-04T03:27:07.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101682025-03-04T03:27:07.333ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
101692025-03-04T03:27:07.333ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
101702025-03-04T03:27:07.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
101712025-03-04T03:27:07.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
101722025-03-04T03:27:07.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
101732025-03-04T03:27:07.336ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101742025-03-04T03:27:07.337ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 126.944726ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
101752025-03-04T03:27:07.339ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
101762025-03-04T03:27:07.339ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
101772025-03-04T03:27:07.339ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
101782025-03-04T03:27:07.339ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 134.66212ms iteration = 10 status = {"generation":1}
101792025-03-04T03:27:07.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
101802025-03-04T03:27:07.339ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101812025-03-04T03:27:07.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101822025-03-04T03:27:07.342ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
101832025-03-04T03:27:07.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
101842025-03-04T03:27:07.367ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
101852025-03-04T03:27:07.367ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 163.501164ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
101862025-03-04T03:27:07.369ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
101872025-03-04T03:27:07.369ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
101882025-03-04T03:27:07.369ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101892025-03-04T03:27:07.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101902025-03-04T03:27:07.372ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101912025-03-04T03:27:07.377ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
101922025-03-04T03:27:07.377ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101932025-03-04T03:27:07.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101942025-03-04T03:27:07.377ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
101952025-03-04T03:27:07.377ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 172.528505ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
101962025-03-04T03:27:07.379ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
101972025-03-04T03:27:07.379ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
101982025-03-04T03:27:07.379ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
101992025-03-04T03:27:07.379ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 175.224828ms iteration = 10 status = {"generation":2}
102002025-03-04T03:27:07.380ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 169.828211ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
102012025-03-04T03:27:07.419ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
102022025-03-04T03:27:07.419ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 213.710649ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
102032025-03-04T03:27:07.433ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
102042025-03-04T03:27:07.438ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 233.288289ms iteration = 21 status = {"physical_disks_added":0}
102052025-03-04T03:27:07.463ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102062025-03-04T03:27:07.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102072025-03-04T03:27:07.465ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102082025-03-04T03:27:07.467ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
102092025-03-04T03:27:07.467ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102102025-03-04T03:27:07.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102112025-03-04T03:27:07.468ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102122025-03-04T03:27:07.470ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
102132025-03-04T03:27:07.470ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102142025-03-04T03:27:07.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102152025-03-04T03:27:07.501ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
102162025-03-04T03:27:07.522ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
102172025-03-04T03:27:07.522ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
102182025-03-04T03:27:07.522ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2159d3f2-6be7-4a86-9d61-c9a9f8c3c129", "content-length": "26", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
102192025-03-04T03:27:07.522ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
102202025-03-04T03:27:07.523ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
102212025-03-04T03:27:07.529ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
102222025-03-04T03:27:07.529ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
102232025-03-04T03:27:07.529ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2802983-e58b-4a64-be9f-ad73942b6a7a", "content-length": "372", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
102242025-03-04T03:27:07.529ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102252025-03-04T03:27:07.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102262025-03-04T03:27:07.546ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
102272025-03-04T03:27:07.546ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
102282025-03-04T03:27:07.546ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4d2f450d-5255-44d0-bdbb-4534abe0dbf6", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
102292025-03-04T03:27:07.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102302025-03-04T03:27:07.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102312025-03-04T03:27:07.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102322025-03-04T03:27:07.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102332025-03-04T03:27:07.550ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
102342025-03-04T03:27:07.550ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
102352025-03-04T03:27:07.551ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b54475fa-9aa4-4ee9-87fc-9cec66b2caec", "content-length": "26", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
102362025-03-04T03:27:07.551ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
102372025-03-04T03:27:07.553ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
102382025-03-04T03:27:07.601ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 396.5369ms iteration = 19 status = {}
102392025-03-04T03:27:07.623ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
102402025-03-04T03:27:07.624ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "067979e8-8115-42f5-a0aa-2363b153872a", "content-length": "691", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
102412025-03-04T03:27:07.624ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
102422025-03-04T03:27:07.624ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
102432025-03-04T03:27:07.624ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
102442025-03-04T03:27:07.625ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 419.926525ms iteration = 19 status = {}
102452025-03-04T03:27:07.626ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102462025-03-04T03:27:07.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102472025-03-04T03:27:07.630ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
102482025-03-04T03:27:07.630ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
102492025-03-04T03:27:07.630ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "569de924-1b31-45f7-9f56-b3eea275eb57", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
102502025-03-04T03:27:07.632ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 426.942252ms iteration = 19 status = {}
102512025-03-04T03:27:07.633ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
102522025-03-04T03:27:07.633ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2efad164-ae74-4623-ab6e-14686065f2fa", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
102532025-03-04T03:27:07.633ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
102542025-03-04T03:27:07.634ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
102552025-03-04T03:27:07.634ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8717d7f2-ef31-43a3-8360-67b66c36ccc8", "content-length": "1", "date": "Tue, 04 Mar 2025 03:27:06 GMT"} })
102562025-03-04T03:27:07.635ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "8e2117fb-cea7-4b6e-8c07-49189a666cd1", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
102572025-03-04T03:27:07.635ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
102582025-03-04T03:27:07.636ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
102592025-03-04T03:27:07.636ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
102602025-03-04T03:27:07.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
102612025-03-04T03:27:07.636ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c0904b8b-679d-4442-a101-7a017b6cc6d7", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
102622025-03-04T03:27:07.636ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
102632025-03-04T03:27:07.637ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
102642025-03-04T03:27:07.637ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 433.33442ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
102652025-03-04T03:27:07.638ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "4792ee75-4970-4414-893b-6d770be07b84", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
102662025-03-04T03:27:07.638ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
102672025-03-04T03:27:07.638ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 434.223501ms iteration = 19 status = null
102682025-03-04T03:27:07.639ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a173e61c-cc24-4e84-956d-5ba53648013f", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
102692025-03-04T03:27:07.639ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
102702025-03-04T03:27:07.640ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
102712025-03-04T03:27:07.640ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 436.485119ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
102722025-03-04T03:27:07.704ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102732025-03-04T03:27:07.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102742025-03-04T03:27:07.706ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
102752025-03-04T03:27:07.706ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102762025-03-04T03:27:07.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102772025-03-04T03:27:07.717ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
102782025-03-04T03:27:07.717ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "58f4089a-8681-49ee-9c06-f8316e9125fe", "content-length": "134", "date": "Tue, 04 Mar 2025 03:27:06 GMT"} })
102792025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
102802025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
102812025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
102822025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
102832025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
102842025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
102852025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
102862025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
102872025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
102882025-03-04T03:27:07.717ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
102892025-03-04T03:27:07.718ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
102902025-03-04T03:27:07.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102912025-03-04T03:27:07.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102922025-03-04T03:27:07.720ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
102932025-03-04T03:27:07.720ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6fbc8d3-c6c7-4467-a0a9-10c30e8e52b9", "content-length": "2", "date": "Tue, 04 Mar 2025 03:27:06 GMT"} })
102942025-03-04T03:27:07.720ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
102952025-03-04T03:27:07.720ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "77d7e7ef-5dc0-43c6-b43e-dd166e2e8040", "content-length": "51", "date": "Tue, 04 Mar 2025 03:27:06 GMT"} })
102962025-03-04T03:27:07.720ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
102972025-03-04T03:27:07.720ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
102982025-03-04T03:27:07.812ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2af2fd19-5957-4833-bc75-d103f61b3d99", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
102992025-03-04T03:27:07.812ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
103002025-03-04T03:27:07.812ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
103012025-03-04T03:27:07.812ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5564923f-b57b-4d33-a08b-40651ec4a63e", "content-length": "682", "date": "Tue, 04 Mar 2025 03:27:07 GMT"} })
103022025-03-04T03:27:07.812ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103032025-03-04T03:27:07.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103042025-03-04T03:27:07.816ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103052025-03-04T03:27:07.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103062025-03-04T03:27:07.820ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103072025-03-04T03:27:07.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103082025-03-04T03:27:07.822ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103092025-03-04T03:27:07.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103102025-03-04T03:27:07.824ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cf630de-7e36-4373-91ec-189420fee672 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103112025-03-04T03:27:07.824ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cf630de-7e36-4373-91ec-189420fee672 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103122025-03-04T03:27:07.826ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2323 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4cf630de-7e36-4373-91ec-189420fee672 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103132025-03-04T03:27:07.829ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 624.102319ms iteration = 19 status = {}
103142025-03-04T03:27:07.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 459 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = b67746cc-bf84-4fc9-a25a-0db86f408066 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
103152025-03-04T03:27:08.845ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cf3f5da4-3e19-4815-abc8-7d8c8084b14e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103162025-03-04T03:27:08.845ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cf3f5da4-3e19-4815-abc8-7d8c8084b14e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103172025-03-04T03:27:08.847ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2204 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cf3f5da4-3e19-4815-abc8-7d8c8084b14e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103182025-03-04T03:27:09.873ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c1afe23-549d-4851-aea4-f1e084b7b72d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103192025-03-04T03:27:09.873ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c1afe23-549d-4851-aea4-f1e084b7b72d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103202025-03-04T03:27:09.876ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2658 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c1afe23-549d-4851-aea4-f1e084b7b72d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103212025-03-04T03:27:10.897ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8cf3e8e0-dbdf-42b1-a713-b7728b646c4d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103222025-03-04T03:27:10.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8cf3e8e0-dbdf-42b1-a713-b7728b646c4d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103232025-03-04T03:27:10.899ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2322 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8cf3e8e0-dbdf-42b1-a713-b7728b646c4d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103242025-03-04T03:27:11.919ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8897f4cf-41f3-4fa1-905b-ebf7321a3581 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103252025-03-04T03:27:11.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8897f4cf-41f3-4fa1-905b-ebf7321a3581 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103262025-03-04T03:27:11.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2413 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8897f4cf-41f3-4fa1-905b-ebf7321a3581 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103272025-03-04T03:27:12.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bac94292-18d3-4625-80d0-6cdf3b0026af roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103282025-03-04T03:27:12.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bac94292-18d3-4625-80d0-6cdf3b0026af resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103292025-03-04T03:27:12.942ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2007 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bac94292-18d3-4625-80d0-6cdf3b0026af response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103302025-03-04T03:27:13.961ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3369bfb5-30c3-485a-adf0-c9b6f0647c97 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103312025-03-04T03:27:13.962ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3369bfb5-30c3-485a-adf0-c9b6f0647c97 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103322025-03-04T03:27:13.963ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2040 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3369bfb5-30c3-485a-adf0-c9b6f0647c97 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103332025-03-04T03:27:14.983ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7deac5a0-7094-4ec9-81d8-3022cbe75321 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103342025-03-04T03:27:14.983ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7deac5a0-7094-4ec9-81d8-3022cbe75321 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103352025-03-04T03:27:14.985ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2582 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7deac5a0-7094-4ec9-81d8-3022cbe75321 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103362025-03-04T03:27:16.006ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 955b225d-0ad5-4f40-a672-81265e6f618b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103372025-03-04T03:27:16.006ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 955b225d-0ad5-4f40-a672-81265e6f618b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103382025-03-04T03:27:16.008ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2279 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 955b225d-0ad5-4f40-a672-81265e6f618b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103392025-03-04T03:27:17.028ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 791d80b0-f0f0-402b-aaf5-f6569ae0a1f4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103402025-03-04T03:27:17.028ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 791d80b0-f0f0-402b-aaf5-f6569ae0a1f4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103412025-03-04T03:27:17.030ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2134 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 791d80b0-f0f0-402b-aaf5-f6569ae0a1f4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103422025-03-04T03:27:17.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
103432025-03-04T03:27:17.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103442025-03-04T03:27:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103452025-03-04T03:27:17.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
103462025-03-04T03:27:17.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103472025-03-04T03:27:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103482025-03-04T03:27:17.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103492025-03-04T03:27:17.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103502025-03-04T03:27:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103512025-03-04T03:27:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
103522025-03-04T03:27:17.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103532025-03-04T03:27:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103542025-03-04T03:27:17.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103552025-03-04T03:27:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103562025-03-04T03:27:17.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103572025-03-04T03:27:17.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
103582025-03-04T03:27:17.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103592025-03-04T03:27:17.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103602025-03-04T03:27:17.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
103612025-03-04T03:27:17.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.116168ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
103622025-03-04T03:27:17.237ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.826308ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
103632025-03-04T03:27:17.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 468 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = e7df9493-cbd5-4d89-8835-4642f994faf1 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
103642025-03-04T03:27:18.050ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5091dc2-8478-4d6e-860e-0b91d8beed33 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103652025-03-04T03:27:18.050ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5091dc2-8478-4d6e-860e-0b91d8beed33 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103662025-03-04T03:27:18.052ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2471 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b5091dc2-8478-4d6e-860e-0b91d8beed33 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103672025-03-04T03:27:19.070ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca4122b3-5d26-4ba7-b50e-9d7766677403 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103682025-03-04T03:27:19.071ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca4122b3-5d26-4ba7-b50e-9d7766677403 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103692025-03-04T03:27:19.073ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2554 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ca4122b3-5d26-4ba7-b50e-9d7766677403 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103702025-03-04T03:27:20.090ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e44d8457-0cf6-4fae-a190-b61c96f57f0a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103712025-03-04T03:27:20.091ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e44d8457-0cf6-4fae-a190-b61c96f57f0a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103722025-03-04T03:27:20.093ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2582 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e44d8457-0cf6-4fae-a190-b61c96f57f0a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103732025-03-04T03:27:21.113ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4ee21c7e-c7fe-489a-85b9-58d6e700953e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103742025-03-04T03:27:21.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4ee21c7e-c7fe-489a-85b9-58d6e700953e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103752025-03-04T03:27:21.115ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2088 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4ee21c7e-c7fe-489a-85b9-58d6e700953e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103762025-03-04T03:27:22.135ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a84ea4e-845a-47aa-8680-a0e26e50e8d3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103772025-03-04T03:27:22.135ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a84ea4e-845a-47aa-8680-a0e26e50e8d3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103782025-03-04T03:27:22.137ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2455 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a84ea4e-845a-47aa-8680-a0e26e50e8d3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103792025-03-04T03:27:23.156ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dac0347a-0358-4e60-9607-aa0ba5e06855 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103802025-03-04T03:27:23.157ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dac0347a-0358-4e60-9607-aa0ba5e06855 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103812025-03-04T03:27:23.159ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2363 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dac0347a-0358-4e60-9607-aa0ba5e06855 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103822025-03-04T03:27:24.178ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 52ed2f18-827f-44af-afd3-b01201bef3a0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103832025-03-04T03:27:24.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 52ed2f18-827f-44af-afd3-b01201bef3a0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103842025-03-04T03:27:24.180ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2248 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 52ed2f18-827f-44af-afd3-b01201bef3a0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103852025-03-04T03:27:25.199ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f19db0ae-ab23-42d6-b1b7-0e6453093514 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103862025-03-04T03:27:25.199ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f19db0ae-ab23-42d6-b1b7-0e6453093514 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103872025-03-04T03:27:25.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1950 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f19db0ae-ab23-42d6-b1b7-0e6453093514 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103882025-03-04T03:27:26.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bec9909d-8916-45de-a463-3ef6ef02ea01 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103892025-03-04T03:27:26.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bec9909d-8916-45de-a463-3ef6ef02ea01 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103902025-03-04T03:27:26.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2588 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bec9909d-8916-45de-a463-3ef6ef02ea01 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
103912025-03-04T03:27:27.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
103922025-03-04T03:27:27.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103932025-03-04T03:27:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103942025-03-04T03:27:27.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
103952025-03-04T03:27:27.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103962025-03-04T03:27:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103972025-03-04T03:27:27.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103982025-03-04T03:27:27.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103992025-03-04T03:27:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
104002025-03-04T03:27:27.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.815727ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
104012025-03-04T03:27:27.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
104022025-03-04T03:27:27.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104032025-03-04T03:27:27.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104042025-03-04T03:27:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104052025-03-04T03:27:27.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104062025-03-04T03:27:27.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104072025-03-04T03:27:27.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
104082025-03-04T03:27:27.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104092025-03-04T03:27:27.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104102025-03-04T03:27:27.224ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
104112025-03-04T03:27:27.224ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.906152ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
104122025-03-04T03:27:27.241ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5cf5e963-7df5-487c-92b7-adb64111e1d5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104132025-03-04T03:27:27.241ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5cf5e963-7df5-487c-92b7-adb64111e1d5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104142025-03-04T03:27:27.242ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1654 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5cf5e963-7df5-487c-92b7-adb64111e1d5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104152025-03-04T03:27:27.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 460 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 3e3edfdf-1f5c-415f-ae20-e2805d291f50 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
104162025-03-04T03:27:28.262ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b9ea59e3-f463-48bf-96c6-0e638e169ee0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104172025-03-04T03:27:28.263ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b9ea59e3-f463-48bf-96c6-0e638e169ee0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104182025-03-04T03:27:28.265ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2426 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b9ea59e3-f463-48bf-96c6-0e638e169ee0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104192025-03-04T03:27:29.285ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 317a519b-813d-4922-ae20-24d2933ef053 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104202025-03-04T03:27:29.285ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 317a519b-813d-4922-ae20-24d2933ef053 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104212025-03-04T03:27:29.287ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2239 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 317a519b-813d-4922-ae20-24d2933ef053 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104222025-03-04T03:27:30.306ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7bb5415c-488e-4348-b022-569bdf795ad7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104232025-03-04T03:27:30.306ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7bb5415c-488e-4348-b022-569bdf795ad7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104242025-03-04T03:27:30.308ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2155 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7bb5415c-488e-4348-b022-569bdf795ad7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104252025-03-04T03:27:31.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0b68e78b-96c2-42dd-b575-c7ca2e803be2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104262025-03-04T03:27:31.327ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0b68e78b-96c2-42dd-b575-c7ca2e803be2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104272025-03-04T03:27:31.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2199 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0b68e78b-96c2-42dd-b575-c7ca2e803be2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104282025-03-04T03:27:32.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2f5d3cea-04de-44e3-96bd-73080652bb5f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104292025-03-04T03:27:32.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2f5d3cea-04de-44e3-96bd-73080652bb5f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104302025-03-04T03:27:32.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2060 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2f5d3cea-04de-44e3-96bd-73080652bb5f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104312025-03-04T03:27:33.369ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aaba313c-e94d-4069-bab9-c9fa43558de4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104322025-03-04T03:27:33.370ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aaba313c-e94d-4069-bab9-c9fa43558de4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104332025-03-04T03:27:33.372ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2408 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aaba313c-e94d-4069-bab9-c9fa43558de4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104342025-03-04T03:27:34.390ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 668d5146-0bfa-4eb7-9f01-e1defae8826e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104352025-03-04T03:27:34.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 668d5146-0bfa-4eb7-9f01-e1defae8826e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104362025-03-04T03:27:34.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2098 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 668d5146-0bfa-4eb7-9f01-e1defae8826e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104372025-03-04T03:27:35.412ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8fc54e46-5b42-4d35-a75f-6784df88a0c6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104382025-03-04T03:27:35.413ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8fc54e46-5b42-4d35-a75f-6784df88a0c6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104392025-03-04T03:27:35.415ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2650 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8fc54e46-5b42-4d35-a75f-6784df88a0c6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104402025-03-04T03:27:36.434ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e988e87-6b60-4cc7-ba66-c9c90c5788cc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104412025-03-04T03:27:36.435ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e988e87-6b60-4cc7-ba66-c9c90c5788cc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104422025-03-04T03:27:36.436ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1941 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e988e87-6b60-4cc7-ba66-c9c90c5788cc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
104432025-03-04T03:27:37.204ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
104442025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
104452025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
104462025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
104472025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
104482025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
104492025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
104502025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
104512025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
104522025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Timeout
104532025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
104542025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
104552025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
104562025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
104572025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
104582025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
104592025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
104602025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104612025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
104622025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
104632025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104642025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
104652025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
104662025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104672025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
104682025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
104692025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
104702025-03-04T03:27:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104712025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
104722025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
104732025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
104742025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
104752025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
104762025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
104772025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
104782025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
104792025-03-04T03:27:37.206ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
104802025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104812025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
104822025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
104832025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104842025-03-04T03:27:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
104852025-03-04T03:27:37.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1569, tv_nsec: 402467422 } })
104862025-03-04T03:27:37.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
104872025-03-04T03:27:37.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1569, tv_nsec: 402950388 } })
104882025-03-04T03:27:37.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
104892025-03-04T03:27:37.207ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104902025-03-04T03:27:37.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104912025-03-04T03:27:37.208ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104922025-03-04T03:27:37.208ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.494093ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
104932025-03-04T03:27:37.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104942025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
104952025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
104962025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
104972025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
104982025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
104992025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
105002025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
105012025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
105022025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
105032025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
105042025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
105052025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
105062025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
105072025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
105082025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
105092025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
105102025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
105112025-03-04T03:27:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
105122025-03-04T03:27:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
105132025-03-04T03:27:37.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
105142025-03-04T03:27:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
105152025-03-04T03:27:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
105162025-03-04T03:27:37.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105172025-03-04T03:27:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
105182025-03-04T03:27:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
105192025-03-04T03:27:37.212ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105202025-03-04T03:27:37.212ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105212025-03-04T03:27:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105222025-03-04T03:27:37.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105232025-03-04T03:27:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
105242025-03-04T03:27:37.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
105252025-03-04T03:27:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
105262025-03-04T03:27:37.214ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105272025-03-04T03:27:37.214ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
105282025-03-04T03:27:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
105292025-03-04T03:27:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
105302025-03-04T03:27:37.215ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
105312025-03-04T03:27:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
105322025-03-04T03:27:37.216ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
105332025-03-04T03:27:37.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105342025-03-04T03:27:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105352025-03-04T03:27:37.220ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
105362025-03-04T03:27:37.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105372025-03-04T03:27:37.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
105382025-03-04T03:27:37.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
105392025-03-04T03:27:37.220ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105402025-03-04T03:27:37.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105412025-03-04T03:27:37.221ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1569, tv_nsec: 417244453 } })
105422025-03-04T03:27:37.222ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
105432025-03-04T03:27:37.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105442025-03-04T03:27:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
105452025-03-04T03:27:37.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105462025-03-04T03:27:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
105472025-03-04T03:27:37.246ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105482025-03-04T03:27:37.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105492025-03-04T03:27:37.248ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
105502025-03-04T03:27:37.248ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 43.027267ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
105512025-03-04T03:27:37.250ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105522025-03-04T03:27:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105532025-03-04T03:27:37.252ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
105542025-03-04T03:27:37.252ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
105552025-03-04T03:27:37.252ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
105562025-03-04T03:27:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
105572025-03-04T03:27:37.258ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105582025-03-04T03:27:37.259ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 49.197183ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
105592025-03-04T03:27:37.259ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
105602025-03-04T03:27:37.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
105612025-03-04T03:27:37.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
105622025-03-04T03:27:37.262ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105632025-03-04T03:27:37.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105642025-03-04T03:27:37.265ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 55.103705ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
105652025-03-04T03:27:37.265ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 60.072707ms iteration = 20 status = 0
105662025-03-04T03:27:37.265ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 55.236117ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
105672025-03-04T03:27:37.265ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
105682025-03-04T03:27:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
105692025-03-04T03:27:37.267ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 57.390383ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
105702025-03-04T03:27:37.268ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
105712025-03-04T03:27:37.268ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.01827ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
105722025-03-04T03:27:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
105732025-03-04T03:27:37.270ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105742025-03-04T03:27:37.272ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
105752025-03-04T03:27:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
105762025-03-04T03:27:37.272ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
105772025-03-04T03:27:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
105782025-03-04T03:27:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
105792025-03-04T03:27:37.274ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105802025-03-04T03:27:37.275ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105812025-03-04T03:27:37.276ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
105822025-03-04T03:27:37.276ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105832025-03-04T03:27:37.277ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
105842025-03-04T03:27:37.277ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
105852025-03-04T03:27:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105862025-03-04T03:27:37.277ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a560d0f1-382b-400f-8a1c-3ab6b51e61a3", "content-length": "26", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
105872025-03-04T03:27:37.277ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
105882025-03-04T03:27:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
105892025-03-04T03:27:37.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
105902025-03-04T03:27:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
105912025-03-04T03:27:37.279ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
105922025-03-04T03:27:37.280ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
105932025-03-04T03:27:37.280ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 73.641273ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
105942025-03-04T03:27:37.290ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
105952025-03-04T03:27:37.290ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
105962025-03-04T03:27:37.290ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
105972025-03-04T03:27:37.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
105982025-03-04T03:27:37.297ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
105992025-03-04T03:27:37.297ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
106002025-03-04T03:27:37.298ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e5dd834-89b4-4dd6-b3f1-fc0e8f870736", "content-length": "26", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
106012025-03-04T03:27:37.298ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
106022025-03-04T03:27:37.300ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 94.733581ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
106032025-03-04T03:27:37.300ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
106042025-03-04T03:27:37.305ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
106052025-03-04T03:27:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
106062025-03-04T03:27:37.308ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 97.718448ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
106072025-03-04T03:27:37.309ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
106082025-03-04T03:27:37.309ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 104.570421ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
106092025-03-04T03:27:37.316ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 110.663356ms iteration = 22 status = {"physical_disks_added":0}
106102025-03-04T03:27:37.321ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
106112025-03-04T03:27:37.321ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
106122025-03-04T03:27:37.321ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 115.108441ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
106132025-03-04T03:27:37.335ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106142025-03-04T03:27:37.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106152025-03-04T03:27:37.348ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
106162025-03-04T03:27:37.348ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
106172025-03-04T03:27:37.348ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "31d2f52f-3603-4310-b003-76eefa4e7c0f", "content-length": "26", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
106182025-03-04T03:27:37.348ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
106192025-03-04T03:27:37.350ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
106202025-03-04T03:27:37.422ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
106212025-03-04T03:27:37.422ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dd18072-f02a-4eac-a722-9e19952dc296", "content-length": "1", "date": "Tue, 04 Mar 2025 03:27:36 GMT"} })
106222025-03-04T03:27:37.424ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
106232025-03-04T03:27:37.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
106242025-03-04T03:27:37.425ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106252025-03-04T03:27:37.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106262025-03-04T03:27:37.449ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
106272025-03-04T03:27:37.449ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
106282025-03-04T03:27:37.449ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8aa169d-f7aa-420c-8438-b0b04a51660f", "content-length": "372", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
106292025-03-04T03:27:37.449ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106302025-03-04T03:27:37.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106312025-03-04T03:27:37.451ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 246.101226ms iteration = 20 status = {}
106322025-03-04T03:27:37.451ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 246.213877ms iteration = 20 status = null
106332025-03-04T03:27:37.451ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106342025-03-04T03:27:37.452ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
106352025-03-04T03:27:37.452ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106362025-03-04T03:27:37.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106372025-03-04T03:27:37.454ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106382025-03-04T03:27:37.456ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
106392025-03-04T03:27:37.456ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106402025-03-04T03:27:37.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106412025-03-04T03:27:37.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d5d57a92-8353-4059-b344-acf02ad10fd7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
106422025-03-04T03:27:37.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d5d57a92-8353-4059-b344-acf02ad10fd7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
106432025-03-04T03:27:37.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1584 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d5d57a92-8353-4059-b344-acf02ad10fd7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
106442025-03-04T03:27:37.492ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
106452025-03-04T03:27:37.492ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
106462025-03-04T03:27:37.492ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4acf2e76-17f3-4db0-bf74-b102193d208b", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
106472025-03-04T03:27:37.492ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106482025-03-04T03:27:37.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106492025-03-04T03:27:37.495ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106502025-03-04T03:27:37.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106512025-03-04T03:27:37.498ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106522025-03-04T03:27:37.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106532025-03-04T03:27:37.501ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
106542025-03-04T03:27:37.501ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
106552025-03-04T03:27:37.501ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "49f973bf-5bb7-4941-aa4c-627b33ba4772", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
106562025-03-04T03:27:37.503ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 297.363565ms iteration = 20 status = {}
106572025-03-04T03:27:37.515ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
106582025-03-04T03:27:37.515ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "297dbdc2-d081-4605-8e5a-a20608f11a83", "content-length": "691", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
106592025-03-04T03:27:37.515ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
106602025-03-04T03:27:37.515ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
106612025-03-04T03:27:37.515ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
106622025-03-04T03:27:37.516ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 310.87896ms iteration = 20 status = {}
106632025-03-04T03:27:37.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106642025-03-04T03:27:37.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106652025-03-04T03:27:37.555ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
106662025-03-04T03:27:37.555ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106672025-03-04T03:27:37.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106682025-03-04T03:27:37.564ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
106692025-03-04T03:27:37.565ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee6bb1af-df41-460e-97c7-642cd4063532", "content-length": "134", "date": "Tue, 04 Mar 2025 03:27:36 GMT"} })
106702025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
106712025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
106722025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
106732025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
106742025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
106752025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
106762025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
106772025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
106782025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
106792025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
106802025-03-04T03:27:37.565ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
106812025-03-04T03:27:37.565ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106822025-03-04T03:27:37.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106832025-03-04T03:27:37.567ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
106842025-03-04T03:27:37.567ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0292e91-f1b3-494d-abd6-1f657955f2c4", "content-length": "2", "date": "Tue, 04 Mar 2025 03:27:36 GMT"} })
106852025-03-04T03:27:37.567ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
106862025-03-04T03:27:37.567ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a86c02c-bf36-4395-8ae9-daa2a4e08e48", "content-length": "51", "date": "Tue, 04 Mar 2025 03:27:36 GMT"} })
106872025-03-04T03:27:37.567ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
106882025-03-04T03:27:37.567ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
106892025-03-04T03:27:37.660ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ac0dd6a3-a8f0-40b4-8a44-feaa4f1b9109", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
106902025-03-04T03:27:37.660ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
106912025-03-04T03:27:37.660ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
106922025-03-04T03:27:37.660ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f85511c-3839-4375-bc06-522ac35b5e43", "content-length": "682", "date": "Tue, 04 Mar 2025 03:27:37 GMT"} })
106932025-03-04T03:27:37.660ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106942025-03-04T03:27:37.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106952025-03-04T03:27:37.664ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106962025-03-04T03:27:37.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106972025-03-04T03:27:37.668ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106982025-03-04T03:27:37.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106992025-03-04T03:27:37.670ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107002025-03-04T03:27:37.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107012025-03-04T03:27:37.678ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 472.727053ms iteration = 20 status = {}
107022025-03-04T03:27:37.896ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 465 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = adfb8ee6-97da-4ab9-a3cf-4055093edc63 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
107032025-03-04T03:27:38.478ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2f96f7d2-6614-466e-a6ef-0cf5a6234df9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107042025-03-04T03:27:38.479ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2f96f7d2-6614-466e-a6ef-0cf5a6234df9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107052025-03-04T03:27:38.484ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 6102 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2f96f7d2-6614-466e-a6ef-0cf5a6234df9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107062025-03-04T03:27:39.506ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bda1fd1f-fe1e-4182-9823-70698bd01acb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107072025-03-04T03:27:39.506ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bda1fd1f-fe1e-4182-9823-70698bd01acb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107082025-03-04T03:27:39.508ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2303 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bda1fd1f-fe1e-4182-9823-70698bd01acb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107092025-03-04T03:27:40.527ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51b7551b-dae8-4b46-8604-c13f37968354 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107102025-03-04T03:27:40.527ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51b7551b-dae8-4b46-8604-c13f37968354 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107112025-03-04T03:27:40.529ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1944 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51b7551b-dae8-4b46-8604-c13f37968354 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107122025-03-04T03:27:41.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 56f0e6b3-093a-4e37-89b2-2038b9daa86b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107132025-03-04T03:27:41.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 56f0e6b3-093a-4e37-89b2-2038b9daa86b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107142025-03-04T03:27:41.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 56f0e6b3-093a-4e37-89b2-2038b9daa86b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107152025-03-04T03:27:42.570ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 169871c7-88f4-4f42-b637-cb907076b3c5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107162025-03-04T03:27:42.570ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 169871c7-88f4-4f42-b637-cb907076b3c5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107172025-03-04T03:27:42.572ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2251 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 169871c7-88f4-4f42-b637-cb907076b3c5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107182025-03-04T03:27:43.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c24194c4-b4b9-468a-adaa-f448bf9087a5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107192025-03-04T03:27:43.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c24194c4-b4b9-468a-adaa-f448bf9087a5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107202025-03-04T03:27:43.595ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3165 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c24194c4-b4b9-468a-adaa-f448bf9087a5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107212025-03-04T03:27:44.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93e90d60-e5a6-49ff-a64c-b8d2d50d56a7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107222025-03-04T03:27:44.620ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93e90d60-e5a6-49ff-a64c-b8d2d50d56a7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107232025-03-04T03:27:44.622ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2624 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 93e90d60-e5a6-49ff-a64c-b8d2d50d56a7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107242025-03-04T03:27:45.643ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b71d65ec-9f2e-4111-b653-f0f08fd17fea roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107252025-03-04T03:27:45.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b71d65ec-9f2e-4111-b653-f0f08fd17fea resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107262025-03-04T03:27:45.645ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1858 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b71d65ec-9f2e-4111-b653-f0f08fd17fea response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107272025-03-04T03:27:46.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3985d624-cc85-4b51-97bb-c82fc0273075 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107282025-03-04T03:27:46.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3985d624-cc85-4b51-97bb-c82fc0273075 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107292025-03-04T03:27:46.667ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2825 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3985d624-cc85-4b51-97bb-c82fc0273075 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107302025-03-04T03:27:47.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
107312025-03-04T03:27:47.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107322025-03-04T03:27:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107332025-03-04T03:27:47.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
107342025-03-04T03:27:47.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
107352025-03-04T03:27:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
107362025-03-04T03:27:47.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107372025-03-04T03:27:47.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
107382025-03-04T03:27:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
107392025-03-04T03:27:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
107402025-03-04T03:27:47.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107412025-03-04T03:27:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107422025-03-04T03:27:47.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107432025-03-04T03:27:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107442025-03-04T03:27:47.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107452025-03-04T03:27:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
107462025-03-04T03:27:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107472025-03-04T03:27:47.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107482025-03-04T03:27:47.225ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
107492025-03-04T03:27:47.225ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.214887ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
107502025-03-04T03:27:47.237ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.631918ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
107512025-03-04T03:27:47.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d7cd17d-5db1-4d97-825d-1ea437e55a89 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107522025-03-04T03:27:47.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d7cd17d-5db1-4d97-825d-1ea437e55a89 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107532025-03-04T03:27:47.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2425 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d7cd17d-5db1-4d97-825d-1ea437e55a89 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107542025-03-04T03:27:47.891ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:62053
107552025-03-04T03:27:47.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = d757d5dc-58ba-408b-832e-b8643c59f633 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
107562025-03-04T03:27:47.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = d757d5dc-58ba-408b-832e-b8643c59f633 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
107572025-03-04T03:27:47.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2709 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = d757d5dc-58ba-408b-832e-b8643c59f633 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
107582025-03-04T03:27:47.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 3ad97c5b-d623-4321-90af-101e5e46ed95 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
107592025-03-04T03:27:47.894ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 3ad97c5b-d623-4321-90af-101e5e46ed95 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
107602025-03-04T03:27:47.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 619 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 62058b08-7f9c-4b22-95f5-124a4a87dac2 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
107612025-03-04T03:27:47.898ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4577 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 3ad97c5b-d623-4321-90af-101e5e46ed95 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
107622025-03-04T03:27:48.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88bf3db9-8228-49a5-b3e1-498ca960140d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107632025-03-04T03:27:48.708ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88bf3db9-8228-49a5-b3e1-498ca960140d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107642025-03-04T03:27:48.710ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2523 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 88bf3db9-8228-49a5-b3e1-498ca960140d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107652025-03-04T03:27:49.729ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb8344ec-38a1-4b2d-b0ee-99fe95504969 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107662025-03-04T03:27:49.730ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb8344ec-38a1-4b2d-b0ee-99fe95504969 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107672025-03-04T03:27:49.732ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2599 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb8344ec-38a1-4b2d-b0ee-99fe95504969 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107682025-03-04T03:27:50.750ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef53bddd-fbae-413a-aec7-4cda11c635f8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107692025-03-04T03:27:50.751ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef53bddd-fbae-413a-aec7-4cda11c635f8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107702025-03-04T03:27:50.752ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2055 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef53bddd-fbae-413a-aec7-4cda11c635f8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107712025-03-04T03:27:51.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8969cb6d-f9d4-412e-8d06-fd21bfa07271 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107722025-03-04T03:27:51.772ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8969cb6d-f9d4-412e-8d06-fd21bfa07271 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107732025-03-04T03:27:51.774ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2290 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8969cb6d-f9d4-412e-8d06-fd21bfa07271 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107742025-03-04T03:27:52.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 27a33933-b2c4-41f5-a428-84fafdd8a48e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107752025-03-04T03:27:52.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 27a33933-b2c4-41f5-a428-84fafdd8a48e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107762025-03-04T03:27:52.796ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2000 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 27a33933-b2c4-41f5-a428-84fafdd8a48e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107772025-03-04T03:27:53.817ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7794b1f-2f74-4ce6-bfc5-8580ac066f66 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107782025-03-04T03:27:53.818ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7794b1f-2f74-4ce6-bfc5-8580ac066f66 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107792025-03-04T03:27:53.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1894 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7794b1f-2f74-4ce6-bfc5-8580ac066f66 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107802025-03-04T03:27:54.840ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d30c6554-5a80-4dc4-8b93-39e0392975b2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107812025-03-04T03:27:54.840ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d30c6554-5a80-4dc4-8b93-39e0392975b2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107822025-03-04T03:27:54.842ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2316 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d30c6554-5a80-4dc4-8b93-39e0392975b2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107832025-03-04T03:27:55.861ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8d7a9d21-2b1e-4ae9-b754-093c0327ddaa roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107842025-03-04T03:27:55.862ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8d7a9d21-2b1e-4ae9-b754-093c0327ddaa resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107852025-03-04T03:27:55.863ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2229 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8d7a9d21-2b1e-4ae9-b754-093c0327ddaa response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107862025-03-04T03:27:56.883ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6cc2d153-756b-41d1-89f0-5f8754e9ca6e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107872025-03-04T03:27:56.883ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6cc2d153-756b-41d1-89f0-5f8754e9ca6e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107882025-03-04T03:27:56.886ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2844 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6cc2d153-756b-41d1-89f0-5f8754e9ca6e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
107892025-03-04T03:27:57.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
107902025-03-04T03:27:57.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107912025-03-04T03:27:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107922025-03-04T03:27:57.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
107932025-03-04T03:27:57.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
107942025-03-04T03:27:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
107952025-03-04T03:27:57.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107962025-03-04T03:27:57.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
107972025-03-04T03:27:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
107982025-03-04T03:27:57.210ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.665535ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
107992025-03-04T03:27:57.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
108002025-03-04T03:27:57.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
108012025-03-04T03:27:57.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
108022025-03-04T03:27:57.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
108032025-03-04T03:27:57.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
108042025-03-04T03:27:57.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108052025-03-04T03:27:57.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
108062025-03-04T03:27:57.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
108072025-03-04T03:27:57.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
108082025-03-04T03:27:57.228ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
108092025-03-04T03:27:57.228ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.04321ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
108102025-03-04T03:27:57.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 731 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = fd4d4185-4126-44bb-a051-4a0d1732d0f3 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
108112025-03-04T03:27:57.905ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a2249ad4-5419-4a5e-9318-33bfea761280 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108122025-03-04T03:27:57.906ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a2249ad4-5419-4a5e-9318-33bfea761280 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108132025-03-04T03:27:57.907ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1834 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a2249ad4-5419-4a5e-9318-33bfea761280 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108142025-03-04T03:27:58.926ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c3cf684-6d12-409a-b251-b4fde2f43ab3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108152025-03-04T03:27:58.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c3cf684-6d12-409a-b251-b4fde2f43ab3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108162025-03-04T03:27:58.928ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1879 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8c3cf684-6d12-409a-b251-b4fde2f43ab3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108172025-03-04T03:27:59.947ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3da69077-d7aa-4ec0-bf9f-715773598e86 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108182025-03-04T03:27:59.947ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3da69077-d7aa-4ec0-bf9f-715773598e86 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108192025-03-04T03:27:59.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2652 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3da69077-d7aa-4ec0-bf9f-715773598e86 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108202025-03-04T03:28:00.969ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 98680f7c-b914-48ec-88bf-d11512dda6c7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108212025-03-04T03:28:00.969ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 98680f7c-b914-48ec-88bf-d11512dda6c7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108222025-03-04T03:28:00.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2423 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 98680f7c-b914-48ec-88bf-d11512dda6c7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108232025-03-04T03:28:01.990ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1192d298-c3b1-4d73-b3ab-a8c13c6ed5f1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108242025-03-04T03:28:01.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1192d298-c3b1-4d73-b3ab-a8c13c6ed5f1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108252025-03-04T03:28:01.992ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2225 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1192d298-c3b1-4d73-b3ab-a8c13c6ed5f1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108262025-03-04T03:28:03.011ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 437f9ff5-0283-491c-b89f-e5d4f56e3281 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108272025-03-04T03:28:03.011ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 437f9ff5-0283-491c-b89f-e5d4f56e3281 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108282025-03-04T03:28:03.013ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1753 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 437f9ff5-0283-491c-b89f-e5d4f56e3281 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108292025-03-04T03:28:04.033ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d303b239-c994-4b06-99d2-1a423051098a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108302025-03-04T03:28:04.034ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d303b239-c994-4b06-99d2-1a423051098a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108312025-03-04T03:28:04.036ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2871 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d303b239-c994-4b06-99d2-1a423051098a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108322025-03-04T03:28:05.055ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb2b13f9-f841-415a-a9ea-77e0a1840a42 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108332025-03-04T03:28:05.055ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb2b13f9-f841-415a-a9ea-77e0a1840a42 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108342025-03-04T03:28:05.057ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2592 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bb2b13f9-f841-415a-a9ea-77e0a1840a42 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108352025-03-04T03:28:06.077ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 597b3aa7-1938-4efc-9a5e-70f8f01108c5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108362025-03-04T03:28:06.078ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 597b3aa7-1938-4efc-9a5e-70f8f01108c5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108372025-03-04T03:28:06.079ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2187 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 597b3aa7-1938-4efc-9a5e-70f8f01108c5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108382025-03-04T03:28:07.098ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e4939981-c2af-4deb-a783-9d1396996b5c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108392025-03-04T03:28:07.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e4939981-c2af-4deb-a783-9d1396996b5c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108402025-03-04T03:28:07.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1726 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e4939981-c2af-4deb-a783-9d1396996b5c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
108412025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
108422025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
108432025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
108442025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
108452025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
108462025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
108472025-03-04T03:28:07.205ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:18:07.205279055 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
108482025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
108492025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
108502025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
108512025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
108522025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
108532025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
108542025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
108552025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
108562025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
108572025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
108582025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
108592025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
108602025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
108612025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
108622025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
108632025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
108642025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
108652025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
108662025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Timeout
108672025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108682025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
108692025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
108702025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
108712025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108722025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Timeout
108732025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
108742025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
108752025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
108762025-03-04T03:28:07.205ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
108772025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
108782025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
108792025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
108802025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
108812025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
108822025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
108832025-03-04T03:28:07.205ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
108842025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.83\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
108852025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
108862025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
108872025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
108882025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
108892025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
108902025-03-04T03:28:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1599, tv_nsec: 401053928 } })
108912025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
108922025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
108932025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
108942025-03-04T03:28:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
108952025-03-04T03:28:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1599, tv_nsec: 401101248 } })
108962025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
108972025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
108982025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
108992025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109002025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109012025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
109022025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
109032025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
109042025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
109052025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
109062025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.088963ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
109072025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
109082025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
109092025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
109102025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
109112025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
109122025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
109132025-03-04T03:28:07.206ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
109142025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
109152025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109162025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
109172025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
109182025-03-04T03:28:07.206ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
109192025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
109202025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
109212025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.277686ms iteration = 11 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
109222025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
109232025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
109242025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
109252025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
109262025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
109272025-03-04T03:28:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109282025-03-04T03:28:07.208ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
109292025-03-04T03:28:07.208ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
109302025-03-04T03:28:07.208ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
109312025-03-04T03:28:07.208ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
109322025-03-04T03:28:07.208ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
109332025-03-04T03:28:07.208ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
109342025-03-04T03:28:07.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109352025-03-04T03:28:07.208ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
109362025-03-04T03:28:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
109372025-03-04T03:28:07.209ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1599, tv_nsec: 404181396 } })
109382025-03-04T03:28:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
109392025-03-04T03:28:07.209ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.526803ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
109402025-03-04T03:28:07.209ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
109412025-03-04T03:28:07.210ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:18:07.205279055 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
109422025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.743129ms iteration = 11 status = {"expiration":"2025-03-04T03:18:07.205279055Z","pruned":[]}
109432025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
109442025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
109452025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
109462025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
109472025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
109482025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
109492025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
109502025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
109512025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
109522025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
109532025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
109542025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
109552025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
109562025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
109572025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
109582025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
109592025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
109602025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
109612025-03-04T03:28:07.210ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1599, tv_nsec: 406016819 } })
109622025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
109632025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
109642025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
109652025-03-04T03:28:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
109662025-03-04T03:28:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
109672025-03-04T03:28:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
109682025-03-04T03:28:07.211ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
109692025-03-04T03:28:07.212ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
109702025-03-04T03:28:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
109712025-03-04T03:28:07.212ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109722025-03-04T03:28:07.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109732025-03-04T03:28:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109742025-03-04T03:28:07.214ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109752025-03-04T03:28:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
109762025-03-04T03:28:07.215ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
109772025-03-04T03:28:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
109782025-03-04T03:28:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
109792025-03-04T03:28:07.215ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
109802025-03-04T03:28:07.215ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
109812025-03-04T03:28:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
109822025-03-04T03:28:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
109832025-03-04T03:28:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
109842025-03-04T03:28:07.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109852025-03-04T03:28:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
109862025-03-04T03:28:07.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109872025-03-04T03:28:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109882025-03-04T03:28:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109892025-03-04T03:28:07.217ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109902025-03-04T03:28:07.218ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109912025-03-04T03:28:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
109922025-03-04T03:28:07.219ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
109932025-03-04T03:28:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
109942025-03-04T03:28:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
109952025-03-04T03:28:07.220ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
109962025-03-04T03:28:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
109972025-03-04T03:28:07.221ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109982025-03-04T03:28:07.222ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109992025-03-04T03:28:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110002025-03-04T03:28:07.224ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
110012025-03-04T03:28:07.224ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
110022025-03-04T03:28:07.224ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 15.881075ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
110032025-03-04T03:28:07.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
110042025-03-04T03:28:07.224ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
110052025-03-04T03:28:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
110062025-03-04T03:28:07.225ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110072025-03-04T03:28:07.228ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
110082025-03-04T03:28:07.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
110092025-03-04T03:28:07.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
110102025-03-04T03:28:07.230ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110112025-03-04T03:28:07.231ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.848475ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
110122025-03-04T03:28:07.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
110132025-03-04T03:28:07.232ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
110142025-03-04T03:28:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
110152025-03-04T03:28:07.239ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110162025-03-04T03:28:07.240ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
110172025-03-04T03:28:07.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
110182025-03-04T03:28:07.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
110192025-03-04T03:28:07.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
110202025-03-04T03:28:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
110212025-03-04T03:28:07.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110222025-03-04T03:28:07.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110232025-03-04T03:28:07.248ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(04891a77-4557-4e1d-91ec-73cedbb3de6c, 0), (69465f72-82eb-4096-b064-2a2e721c6f40, 0), (f43e3c30-d1e9-4487-b67f-c851f5bb40d3, 0), (ebea1c1f-7f7f-45d0-b2be-3da6745abd56, 0)] collection_id = 04891a77-4557-4e1d-91ec-73cedbb3de6c
110242025-03-04T03:28:07.248ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110252025-03-04T03:28:07.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110262025-03-04T03:28:07.260ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
110272025-03-04T03:28:07.260ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
110282025-03-04T03:28:07.260ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
110292025-03-04T03:28:07.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
110302025-03-04T03:28:07.263ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110312025-03-04T03:28:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110322025-03-04T03:28:07.265ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110332025-03-04T03:28:07.266ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110342025-03-04T03:28:07.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110352025-03-04T03:28:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
110362025-03-04T03:28:07.268ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
110372025-03-04T03:28:07.268ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
110382025-03-04T03:28:07.269ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
110392025-03-04T03:28:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
110402025-03-04T03:28:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
110412025-03-04T03:28:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
110422025-03-04T03:28:07.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110432025-03-04T03:28:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110442025-03-04T03:28:07.273ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
110452025-03-04T03:28:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
110462025-03-04T03:28:07.274ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110472025-03-04T03:28:07.274ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110482025-03-04T03:28:07.275ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
110492025-03-04T03:28:07.275ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 67.192774ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
110502025-03-04T03:28:07.276ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
110512025-03-04T03:28:07.276ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
110522025-03-04T03:28:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
110532025-03-04T03:28:07.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110542025-03-04T03:28:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
110552025-03-04T03:28:07.279ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
110562025-03-04T03:28:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
110572025-03-04T03:28:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
110582025-03-04T03:28:07.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
110592025-03-04T03:28:07.280ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
110602025-03-04T03:28:07.280ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
110612025-03-04T03:28:07.280ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
110622025-03-04T03:28:07.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
110632025-03-04T03:28:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
110642025-03-04T03:28:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
110652025-03-04T03:28:07.285ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 74.917828ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
110662025-03-04T03:28:07.287ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
110672025-03-04T03:28:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
110682025-03-04T03:28:07.287ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110692025-03-04T03:28:07.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110702025-03-04T03:28:07.288ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 83.321351ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
110712025-03-04T03:28:07.288ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
110722025-03-04T03:28:07.288ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
110732025-03-04T03:28:07.288ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
110742025-03-04T03:28:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
110752025-03-04T03:28:07.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110762025-03-04T03:28:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110772025-03-04T03:28:07.290ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 79.609665ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
110782025-03-04T03:28:07.290ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 84.03112ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
110792025-03-04T03:28:07.291ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
110802025-03-04T03:28:07.291ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 80.913181ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
110812025-03-04T03:28:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
110822025-03-04T03:28:07.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110832025-03-04T03:28:07.292ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
110842025-03-04T03:28:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
110852025-03-04T03:28:07.293ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1599, tv_nsec: 488470139 } })
110862025-03-04T03:28:07.293ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
110872025-03-04T03:28:07.293ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
110882025-03-04T03:28:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
110892025-03-04T03:28:07.294ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
110902025-03-04T03:28:07.294ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
110912025-03-04T03:28:07.294ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
110922025-03-04T03:28:07.294ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
110932025-03-04T03:28:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
110942025-03-04T03:28:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
110952025-03-04T03:28:07.296ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
110962025-03-04T03:28:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
110972025-03-04T03:28:07.299ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
110982025-03-04T03:28:07.299ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
110992025-03-04T03:28:07.299ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 93.900301ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
111002025-03-04T03:28:07.301ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111012025-03-04T03:28:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111022025-03-04T03:28:07.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111032025-03-04T03:28:07.302ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111042025-03-04T03:28:07.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
111052025-03-04T03:28:07.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
111062025-03-04T03:28:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
111072025-03-04T03:28:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
111082025-03-04T03:28:07.305ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
111092025-03-04T03:28:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
111102025-03-04T03:28:07.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111112025-03-04T03:28:07.307ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111122025-03-04T03:28:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111132025-03-04T03:28:07.309ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
111142025-03-04T03:28:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
111152025-03-04T03:28:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
111162025-03-04T03:28:07.310ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
111172025-03-04T03:28:07.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
111182025-03-04T03:28:07.312ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
111192025-03-04T03:28:07.312ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 107.326505ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
111202025-03-04T03:28:07.313ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111212025-03-04T03:28:07.314ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
111222025-03-04T03:28:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
111232025-03-04T03:28:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
111242025-03-04T03:28:07.317ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111252025-03-04T03:28:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111262025-03-04T03:28:07.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111272025-03-04T03:28:07.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111282025-03-04T03:28:07.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111292025-03-04T03:28:07.320ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111302025-03-04T03:28:07.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111312025-03-04T03:28:07.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111322025-03-04T03:28:07.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
111332025-03-04T03:28:07.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
111342025-03-04T03:28:07.323ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
111352025-03-04T03:28:07.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111362025-03-04T03:28:07.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111372025-03-04T03:28:07.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
111382025-03-04T03:28:07.323ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
111392025-03-04T03:28:07.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
111402025-03-04T03:28:07.325ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111412025-03-04T03:28:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111422025-03-04T03:28:07.326ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
111432025-03-04T03:28:07.326ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 121.428608ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
111442025-03-04T03:28:07.327ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 121.737972ms iteration = 21 status = 0
111452025-03-04T03:28:07.327ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111462025-03-04T03:28:07.329ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111472025-03-04T03:28:07.330ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
111482025-03-04T03:28:07.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
111492025-03-04T03:28:07.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
111502025-03-04T03:28:07.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111512025-03-04T03:28:07.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
111522025-03-04T03:28:07.333ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111532025-03-04T03:28:07.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111542025-03-04T03:28:07.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111552025-03-04T03:28:07.334ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111562025-03-04T03:28:07.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
111572025-03-04T03:28:07.337ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
111582025-03-04T03:28:07.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
111592025-03-04T03:28:07.341ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111602025-03-04T03:28:07.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111612025-03-04T03:28:07.343ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
111622025-03-04T03:28:07.343ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
111632025-03-04T03:28:07.343ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 137.02194ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
111642025-03-04T03:28:07.343ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111652025-03-04T03:28:07.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111662025-03-04T03:28:07.346ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111672025-03-04T03:28:07.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111682025-03-04T03:28:07.346ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
111692025-03-04T03:28:07.346ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 140.425232ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
111702025-03-04T03:28:07.348ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 137.401694ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
111712025-03-04T03:28:07.348ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111722025-03-04T03:28:07.349ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 138.485477ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
111732025-03-04T03:28:07.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111742025-03-04T03:28:07.350ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111752025-03-04T03:28:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111762025-03-04T03:28:07.350ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 04891a77-4557-4e1d-91ec-73cedbb3de6c file = nexus/db-queries/src/db/datastore/inventory.rs:1442 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 0 nerrors = 0 nnics = 0 nnvme_disk_firmware = 0 nphysical_disks = 0 nrot_pages = 8 nrots = 2 nsled_agent_zones = 0 nsled_agents = 0 nsps = 2 nzones = 0 nzpools = 0
111772025-03-04T03:28:07.351ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
111782025-03-04T03:28:07.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
111792025-03-04T03:28:07.352ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111802025-03-04T03:28:07.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111812025-03-04T03:28:07.355ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
111822025-03-04T03:28:07.355ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
111832025-03-04T03:28:07.355ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
111842025-03-04T03:28:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
111852025-03-04T03:28:07.357ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111862025-03-04T03:28:07.357ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
111872025-03-04T03:28:07.357ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111882025-03-04T03:28:07.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
111892025-03-04T03:28:07.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
111902025-03-04T03:28:07.361ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111912025-03-04T03:28:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111922025-03-04T03:28:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
111932025-03-04T03:28:07.362ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
111942025-03-04T03:28:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
111952025-03-04T03:28:07.364ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111962025-03-04T03:28:07.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111972025-03-04T03:28:07.365ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
111982025-03-04T03:28:07.365ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
111992025-03-04T03:28:07.365ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
112002025-03-04T03:28:07.365ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 159.95237ms iteration = 11 status = {"generation":1}
112012025-03-04T03:28:07.366ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112022025-03-04T03:28:07.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112032025-03-04T03:28:07.368ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
112042025-03-04T03:28:07.368ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
112052025-03-04T03:28:07.368ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
112062025-03-04T03:28:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
112072025-03-04T03:28:07.368ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(69465f72-82eb-4096-b064-2a2e721c6f40, 0), (f43e3c30-d1e9-4487-b67f-c851f5bb40d3, 0), (ebea1c1f-7f7f-45d0-b2be-3da6745abd56, 0)]
112082025-03-04T03:28:07.369ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1599, tv_nsec: 564639332 } })
112092025-03-04T03:28:07.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112102025-03-04T03:28:07.374ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
112112025-03-04T03:28:07.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112122025-03-04T03:28:07.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112132025-03-04T03:28:07.376ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
112142025-03-04T03:28:07.396ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
112152025-03-04T03:28:07.396ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 191.322305ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
112162025-03-04T03:28:07.400ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
112172025-03-04T03:28:07.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
112182025-03-04T03:28:07.403ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
112192025-03-04T03:28:07.403ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
112202025-03-04T03:28:07.403ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "46c55d5b-7130-466b-936c-67cef1c483f6", "content-length": "26", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112212025-03-04T03:28:07.403ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
112222025-03-04T03:28:07.405ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
112232025-03-04T03:28:07.409ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
112242025-03-04T03:28:07.409ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 202.920297ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
112252025-03-04T03:28:07.442ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
112262025-03-04T03:28:07.458ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 252.878829ms iteration = 23 status = {"physical_disks_added":0}
112272025-03-04T03:28:07.506ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
112282025-03-04T03:28:07.506ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
112292025-03-04T03:28:07.506ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3257a62-2691-499a-8b5a-4a49ffa241bc", "content-length": "26", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112302025-03-04T03:28:07.506ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
112312025-03-04T03:28:07.509ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
112322025-03-04T03:28:07.566ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
112332025-03-04T03:28:07.566ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
112342025-03-04T03:28:07.567ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0faf652-db3b-4933-904b-864c49027d17", "content-length": "26", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112352025-03-04T03:28:07.567ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
112362025-03-04T03:28:07.568ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
112372025-03-04T03:28:07.651ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
112382025-03-04T03:28:07.651ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab0c72a6-a8b6-4771-8ea0-909be696a482", "content-length": "1", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112392025-03-04T03:28:07.652ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
112402025-03-04T03:28:07.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
112412025-03-04T03:28:07.781ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
112422025-03-04T03:28:07.781ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
112432025-03-04T03:28:07.781ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
112442025-03-04T03:28:07.782ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "105cf2e8-ccbc-42bd-8d86-df8712e7d271", "content-length": "476", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112452025-03-04T03:28:07.782ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
112462025-03-04T03:28:07.782ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6db8a9ee-aed7-4384-ad1e-8e91f7e51596", "content-length": "734", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112472025-03-04T03:28:07.782ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
112482025-03-04T03:28:07.782ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0531371c-c128-4606-bb74-72c6832cfb31", "content-length": "111", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112492025-03-04T03:28:07.782ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
112502025-03-04T03:28:07.782ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c18315c7-b255-435d-8c27-af761a924a0a", "content-length": "111", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112512025-03-04T03:28:07.782ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
112522025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0eac45b5-9b0c-41d2-9330-13e1f1e7a1fa", "content-length": "167", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112532025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
112542025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2c49c91-e6b1-4390-a88c-49f1081f8dc2", "content-length": "167", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112552025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
112562025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "26847a2c-495f-4596-979b-2daaee355220", "content-length": "175", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112572025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
112582025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac995bf8-203a-4821-9df8-fc33486ce297", "content-length": "175", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112592025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
112602025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a6c9d88-7e87-4fbe-be5e-fd01ac29f5c8", "content-length": "702", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112612025-03-04T03:28:07.783ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
112622025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "286e0724-40e0-436d-895e-d27ff8f6ac35", "content-length": "727", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112632025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
112642025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0c5de0d-3f9a-4377-9968-ef71e77cdc00", "content-length": "729", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112652025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
112662025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9499f80-2395-471f-b758-87147643dc4f", "content-length": "728", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112672025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
112682025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f3bf253-0a8c-46fa-a132-b11939ec1e77", "content-length": "722", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112692025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
112702025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "86fbb45f-a8c7-4494-a155-d6f83b130765", "content-length": "109", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112712025-03-04T03:28:07.784ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
112722025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "310455a4-c5cf-433d-9255-695461e8800c", "content-length": "109", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112732025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
112742025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1ef9673-c40f-4e33-a03e-5991d552a430", "content-length": "169", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112752025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
112762025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8895011b-1d23-416d-9e43-0bb225adede8", "content-length": "169", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112772025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
112782025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "229dd609-3a3a-482e-bc46-1497a44fa629", "content-length": "178", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112792025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
112802025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ab4ae54-5157-48ff-910e-29940e53d9bb", "content-length": "178", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112812025-03-04T03:28:07.785ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
112822025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5953ba2d-006c-408a-a41e-97bff85f8b20", "content-length": "702", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112832025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
112842025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dd72e36-6a24-45f1-bc5f-caf1a692947f", "content-length": "727", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112852025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
112862025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af4fd710-ba90-4332-97eb-fc9558c8ad94", "content-length": "729", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112872025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
112882025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1675ebd-ef06-4ce2-8019-4d4d387bc619", "content-length": "728", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112892025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112902025-03-04T03:28:07.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112912025-03-04T03:28:07.798ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
112922025-03-04T03:28:07.843ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
112932025-03-04T03:28:07.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
112942025-03-04T03:28:07.850ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 644.862883ms iteration = 21 status = {}
112952025-03-04T03:28:07.853ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
112962025-03-04T03:28:07.853ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
112972025-03-04T03:28:07.853ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec836bdc-725a-4bc6-b158-79caffb714df", "content-length": "372", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
112982025-03-04T03:28:07.853ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
112992025-03-04T03:28:07.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113002025-03-04T03:28:07.854ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
113012025-03-04T03:28:07.855ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "a933cd81-0779-4a72-9fed-7ddc92ddbd11", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
113022025-03-04T03:28:07.855ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
113032025-03-04T03:28:07.855ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113042025-03-04T03:28:07.856ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113052025-03-04T03:28:07.856ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113062025-03-04T03:28:07.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
113072025-03-04T03:28:07.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113082025-03-04T03:28:07.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
113092025-03-04T03:28:07.856ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e452d5d3-cbe3-4999-ad4e-36855153fa5e", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
113102025-03-04T03:28:07.856ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
113112025-03-04T03:28:07.857ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113122025-03-04T03:28:07.857ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 651.97271ms iteration = 21 status = null
113132025-03-04T03:28:07.857ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113142025-03-04T03:28:07.858ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
113152025-03-04T03:28:07.858ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
113162025-03-04T03:28:07.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
113172025-03-04T03:28:07.858ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
113182025-03-04T03:28:07.858ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 653.56057ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
113192025-03-04T03:28:07.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
113202025-03-04T03:28:07.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
113212025-03-04T03:28:07.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
113222025-03-04T03:28:07.859ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
113232025-03-04T03:28:07.859ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
113242025-03-04T03:28:07.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
113252025-03-04T03:28:07.860ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
113262025-03-04T03:28:07.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
113272025-03-04T03:28:07.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
113282025-03-04T03:28:07.862ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113292025-03-04T03:28:07.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113302025-03-04T03:28:07.862ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113312025-03-04T03:28:07.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113322025-03-04T03:28:07.863ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113332025-03-04T03:28:07.863ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
113342025-03-04T03:28:07.863ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
113352025-03-04T03:28:07.863ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
113362025-03-04T03:28:07.863ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 657.848983ms iteration = 11 status = {"generation":2}
113372025-03-04T03:28:07.863ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
113382025-03-04T03:28:07.864ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
113392025-03-04T03:28:07.864ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
113402025-03-04T03:28:07.864ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa9a428d-9365-4763-bb6b-7735fd6ae379", "content-length": "2", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
113412025-03-04T03:28:07.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
113422025-03-04T03:28:07.864ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:615 sled = http://[fd00:1122:3344:101::1]:12345
113432025-03-04T03:28:07.864ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
113442025-03-04T03:28:07.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
113452025-03-04T03:28:07.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
113462025-03-04T03:28:07.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
113472025-03-04T03:28:07.865ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
113482025-03-04T03:28:07.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
113492025-03-04T03:28:07.866ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 655.498254ms iteration = 3 status = {"last_run_counters":{"copy_err":0,"copy_ok":0,"delete_err":0,"delete_ok":0,"list_err":0,"list_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"delete_err":0,"delete_ok":0,"list_err":0,"list_ok":2,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"operation":"list","target_sled":"5525e426-7299-43ab-ad2b-346968b24e1c","time":"2025-03-04T03:23:07.590439585Z"},{"operation":"list","target_sled":"5525e426-7299-43ab-ad2b-346968b24e1c","time":"2025-03-04T03:28:07.864611785Z"}]}
113502025-03-04T03:28:07.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 578 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 572ae908-ed9b-40ff-9045-264252fa32bc response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
113512025-03-04T03:28:07.897ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
113522025-03-04T03:28:07.897ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
113532025-03-04T03:28:07.898ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c81dc3c6-6ba5-47a2-9ffa-82369059e03b", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
113542025-03-04T03:28:07.898ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113552025-03-04T03:28:07.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113562025-03-04T03:28:07.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113572025-03-04T03:28:07.904ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113582025-03-04T03:28:07.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113592025-03-04T03:28:07.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113602025-03-04T03:28:07.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113612025-03-04T03:28:07.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
113622025-03-04T03:28:07.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
113632025-03-04T03:28:07.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
113642025-03-04T03:28:07.907ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113652025-03-04T03:28:07.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113662025-03-04T03:28:07.908ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
113672025-03-04T03:28:07.908ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 703.253299ms iteration = 5 status = {"blueprint_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","inventory_collection_id":"ebea1c1f-7f7f-45d0-b2be-3da6745abd56","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
113682025-03-04T03:28:07.910ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
113692025-03-04T03:28:07.910ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
113702025-03-04T03:28:07.910ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1d1a374c-5dcd-4255-a990-9cdccfc1fac0", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
113712025-03-04T03:28:07.912ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 705.992902ms iteration = 21 status = {}
113722025-03-04T03:28:07.936ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113732025-03-04T03:28:07.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113742025-03-04T03:28:07.970ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113752025-03-04T03:28:07.974ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
113762025-03-04T03:28:07.974ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113772025-03-04T03:28:07.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113782025-03-04T03:28:07.976ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113792025-03-04T03:28:07.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113802025-03-04T03:28:07.987ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
113812025-03-04T03:28:07.988ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "1cb964e9-d344-4fe3-91de-d349a546261f", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
113822025-03-04T03:28:07.988ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
113832025-03-04T03:28:07.990ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "117f0fde-2121-445b-aafb-eed4056a73e8", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
113842025-03-04T03:28:07.990ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
113852025-03-04T03:28:07.992ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c56b9369-717d-4466-b28b-9dcf00931315", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
113862025-03-04T03:28:07.992ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
113872025-03-04T03:28:07.995ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
113882025-03-04T03:28:07.995ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 790.750631ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
113892025-03-04T03:28:08.011ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113902025-03-04T03:28:08.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113912025-03-04T03:28:08.013ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113922025-03-04T03:28:08.017ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
113932025-03-04T03:28:08.017ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113942025-03-04T03:28:08.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113952025-03-04T03:28:08.021ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113962025-03-04T03:28:08.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113972025-03-04T03:28:08.023ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
113982025-03-04T03:28:08.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
113992025-03-04T03:28:08.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114002025-03-04T03:28:08.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114012025-03-04T03:28:08.098ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114022025-03-04T03:28:08.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114032025-03-04T03:28:08.116ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
114042025-03-04T03:28:08.117ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8159888c-5da6-40aa-b245-7d3cd7850ff3", "content-length": "691", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
114052025-03-04T03:28:08.117ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
114062025-03-04T03:28:08.117ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
114072025-03-04T03:28:08.117ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
114082025-03-04T03:28:08.117ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 912.491923ms iteration = 21 status = {}
114092025-03-04T03:28:08.120ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6973b40f-7f09-48b5-800e-4fd1bd1afe96 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
114102025-03-04T03:28:08.120ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6973b40f-7f09-48b5-800e-4fd1bd1afe96 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
114112025-03-04T03:28:08.120ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
114122025-03-04T03:28:08.120ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
114132025-03-04T03:28:08.123ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
114142025-03-04T03:28:08.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114152025-03-04T03:28:08.123ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114162025-03-04T03:28:08.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114172025-03-04T03:28:08.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114182025-03-04T03:28:08.123ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3723 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6973b40f-7f09-48b5-800e-4fd1bd1afe96 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
114192025-03-04T03:28:08.125ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114202025-03-04T03:28:08.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114212025-03-04T03:28:08.126ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114222025-03-04T03:28:08.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114232025-03-04T03:28:08.128ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114242025-03-04T03:28:08.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114252025-03-04T03:28:08.133ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
114262025-03-04T03:28:08.133ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "033dd272-6820-4f8a-a97d-966cfe9a404f", "content-length": "134", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
114272025-03-04T03:28:08.133ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
114282025-03-04T03:28:08.133ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
114292025-03-04T03:28:08.133ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
114302025-03-04T03:28:08.133ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
114312025-03-04T03:28:08.133ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
114322025-03-04T03:28:08.133ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
114332025-03-04T03:28:08.133ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
114342025-03-04T03:28:08.133ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
114352025-03-04T03:28:08.134ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
114362025-03-04T03:28:08.134ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
114372025-03-04T03:28:08.134ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
114382025-03-04T03:28:08.134ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114392025-03-04T03:28:08.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114402025-03-04T03:28:08.136ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
114412025-03-04T03:28:08.136ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d61e2b7-2f05-47f1-895f-b7a6db711c95", "content-length": "2", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
114422025-03-04T03:28:08.136ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
114432025-03-04T03:28:08.136ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e77912a-e591-4314-b597-80ace854bcf2", "content-length": "51", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
114442025-03-04T03:28:08.136ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
114452025-03-04T03:28:08.136ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
114462025-03-04T03:28:08.158ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114472025-03-04T03:28:08.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
114482025-03-04T03:28:08.162ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114492025-03-04T03:28:08.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114502025-03-04T03:28:08.163ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114512025-03-04T03:28:08.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114522025-03-04T03:28:08.165ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114532025-03-04T03:28:08.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114542025-03-04T03:28:08.195ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114552025-03-04T03:28:08.199ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
114562025-03-04T03:28:08.199ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114572025-03-04T03:28:08.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114582025-03-04T03:28:08.207ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 05a86b80-7d75-4c2d-8253-0cdbdce17c69, kind: Service { id: 2afd3cd1-228b-4949-95e6-cb3413cc5ec2 }, name: Name("nexus-2afd3cd1-228b-4949-95e6-cb3413cc5ec2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 61])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3ce82d4d-d64c-418d-9d43-fc9e659efafa, kind: Service { id: c28b34ab-1b39-4325-87ad-c4ccf5b009d5 }, name: Name("nexus-c28b34ab-1b39-4325-87ad-c4ccf5b009d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 509b968a-6934-4b81-be1c-676fe22e2948, kind: Service { id: d8484452-7d43-42ac-a588-575b6d31b882 }, name: Name("nexus-d8484452-7d43-42ac-a588-575b6d31b882"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 93])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 1efc0800-e2a1-43fb-8159-014fa2b2d648, kind: Service { id: 72700cb3-c938-46f0-9b9c-46cf389d975e }, name: Name("external-dns-72700cb3-c938-46f0-9b9c-46cf389d975e"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 246])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6aab6111-ac14-4d7e-808e-18a2dbaec1a3, kind: Service { id: 0e2dad65-6466-48b5-bc3a-8a9f5178c9b7 }, name: Name("external-dns-0e2dad65-6466-48b5-bc3a-8a9f5178c9b7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
114592025-03-04T03:28:08.207ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114602025-03-04T03:28:08.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114612025-03-04T03:28:08.209ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114622025-03-04T03:28:08.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
114632025-03-04T03:28:08.210ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114642025-03-04T03:28:08.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114652025-03-04T03:28:08.213ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
114662025-03-04T03:28:08.213ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1efc0800-e2a1-43fb-8159-014fa2b2d648, kind: Service { id: 72700cb3-c938-46f0-9b9c-46cf389d975e }, name: Name("external-dns-72700cb3-c938-46f0-9b9c-46cf389d975e"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 246])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6aab6111-ac14-4d7e-808e-18a2dbaec1a3, kind: Service { id: 0e2dad65-6466-48b5-bc3a-8a9f5178c9b7 }, name: Name("external-dns-0e2dad65-6466-48b5-bc3a-8a9f5178c9b7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 8])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 05a86b80-7d75-4c2d-8253-0cdbdce17c69, kind: Service { id: 2afd3cd1-228b-4949-95e6-cb3413cc5ec2 }, name: Name("nexus-2afd3cd1-228b-4949-95e6-cb3413cc5ec2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 61])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3ce82d4d-d64c-418d-9d43-fc9e659efafa, kind: Service { id: c28b34ab-1b39-4325-87ad-c4ccf5b009d5 }, name: Name("nexus-c28b34ab-1b39-4325-87ad-c4ccf5b009d5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 509b968a-6934-4b81-be1c-676fe22e2948, kind: Service { id: d8484452-7d43-42ac-a588-575b6d31b882 }, name: Name("nexus-d8484452-7d43-42ac-a588-575b6d31b882"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 93])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
114672025-03-04T03:28:08.213ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
114682025-03-04T03:28:08.241ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "396295f4-9363-419b-8aeb-a1121d5408ab", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
114692025-03-04T03:28:08.241ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
114702025-03-04T03:28:08.241ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
114712025-03-04T03:28:08.241ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "acdc39d9-57a1-4cbd-9269-47ba49e948cd", "content-length": "682", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
114722025-03-04T03:28:08.241ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114732025-03-04T03:28:08.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114742025-03-04T03:28:08.245ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114752025-03-04T03:28:08.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114762025-03-04T03:28:08.248ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114772025-03-04T03:28:08.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114782025-03-04T03:28:08.250ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114792025-03-04T03:28:08.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114802025-03-04T03:28:08.258ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.052981176s iteration = 21 status = {}
114812025-03-04T03:28:08.339ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "776cd427-690f-4b4a-9c3a-4f2b8ba4111d", "content-length": "31973", "date": "Tue, 04 Mar 2025 03:28:07 GMT"} })
114822025-03-04T03:28:08.341ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
114832025-03-04T03:28:08.341ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
114842025-03-04T03:28:08.341ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
114852025-03-04T03:28:08.343ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
114862025-03-04T03:28:08.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
114872025-03-04T03:28:08.366ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 5525e426-7299-43ab-ad2b-346968b24e1c, time_created: 2025-03-04T03:18:30.785539Z, time_modified: 2025-03-04T03:18:30.785539Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
114882025-03-04T03:28:08.366ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
114892025-03-04T03:28:08.366ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114902025-03-04T03:28:08.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114912025-03-04T03:28:08.368ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114922025-03-04T03:28:08.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114932025-03-04T03:28:08.373ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114942025-03-04T03:28:08.375ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
114952025-03-04T03:28:08.375ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
114962025-03-04T03:28:08.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
114972025-03-04T03:28:08.398ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114982025-03-04T03:28:08.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
114992025-03-04T03:28:08.430ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
115002025-03-04T03:28:08.431ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "aacc534b-e5b3-4e25-8777-adc0efda3fbc", "date": "Tue, 04 Mar 2025 03:28:08 GMT"} })
115012025-03-04T03:28:08.433ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
115022025-03-04T03:28:08.433ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 1.227333222s
115032025-03-04T03:28:08.433ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 1.227343003s iteration = 3 status = {}
115042025-03-04T03:28:08.574ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = d75e63ec-3f87-4a20-99bd-6f13284b7b0c file = nexus/db-queries/src/db/datastore/inventory.rs:1010
115052025-03-04T03:28:08.574ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = d75e63ec-3f87-4a20-99bd-6f13284b7b0c time_started = 2025-03-04 03:28:07.781631 UTC
115062025-03-04T03:28:08.574ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Dependency
115072025-03-04T03:28:08.574ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.368779125s iteration = 3 status = {"collection_id":"d75e63ec-3f87-4a20-99bd-6f13284b7b0c","time_done":"2025-03-04 03:28:08.341599 UTC","time_started":"2025-03-04 03:28:07.781631 UTC"}
115082025-03-04T03:28:08.574ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 6 reason = Dependency
115092025-03-04T03:28:08.574ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115102025-03-04T03:28:08.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115112025-03-04T03:28:08.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115122025-03-04T03:28:08.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115132025-03-04T03:28:08.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115142025-03-04T03:28:08.576ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115152025-03-04T03:28:08.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
115162025-03-04T03:28:08.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115172025-03-04T03:28:08.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115182025-03-04T03:28:08.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
115192025-03-04T03:28:08.579ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115202025-03-04T03:28:08.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115212025-03-04T03:28:08.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115222025-03-04T03:28:08.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115232025-03-04T03:28:08.583ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115242025-03-04T03:28:08.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115252025-03-04T03:28:08.587ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115262025-03-04T03:28:08.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
115272025-03-04T03:28:08.590ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115282025-03-04T03:28:08.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115292025-03-04T03:28:08.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115302025-03-04T03:28:08.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115312025-03-04T03:28:08.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115322025-03-04T03:28:08.638ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
115332025-03-04T03:28:08.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115342025-03-04T03:28:08.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115352025-03-04T03:28:08.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115362025-03-04T03:28:08.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
115372025-03-04T03:28:08.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115382025-03-04T03:28:08.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115392025-03-04T03:28:08.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115402025-03-04T03:28:08.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115412025-03-04T03:28:08.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115422025-03-04T03:28:08.648ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 74.747056ms iteration = 24 status = {"physical_disks_added":0}
115432025-03-04T03:28:08.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
115442025-03-04T03:28:08.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115452025-03-04T03:28:08.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115462025-03-04T03:28:08.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115472025-03-04T03:28:08.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
115482025-03-04T03:28:08.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
115492025-03-04T03:28:08.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
115502025-03-04T03:28:08.655ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
115512025-03-04T03:28:08.655ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 80.83717ms iteration = 6 status = {"blueprint_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","inventory_collection_id":"d75e63ec-3f87-4a20-99bd-6f13284b7b0c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
115522025-03-04T03:28:09.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1812668e-8a04-4484-803e-8c24f5d0fde2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115532025-03-04T03:28:09.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1812668e-8a04-4484-803e-8c24f5d0fde2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115542025-03-04T03:28:09.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2462 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1812668e-8a04-4484-803e-8c24f5d0fde2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115552025-03-04T03:28:10.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f371433-2fb4-4714-966b-6735b509e4fb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115562025-03-04T03:28:10.278ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f371433-2fb4-4714-966b-6735b509e4fb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115572025-03-04T03:28:10.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2269 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f371433-2fb4-4714-966b-6735b509e4fb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115582025-03-04T03:28:11.300ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d6bd5f8-231f-4a79-b036-15895ae242d8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115592025-03-04T03:28:11.300ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d6bd5f8-231f-4a79-b036-15895ae242d8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115602025-03-04T03:28:11.302ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2365 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d6bd5f8-231f-4a79-b036-15895ae242d8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115612025-03-04T03:28:12.321ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25a0104d-bad2-4fc6-b3e1-2ba4d7a5877c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115622025-03-04T03:28:12.321ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25a0104d-bad2-4fc6-b3e1-2ba4d7a5877c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115632025-03-04T03:28:12.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2846 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 25a0104d-bad2-4fc6-b3e1-2ba4d7a5877c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115642025-03-04T03:28:13.343ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5865b651-6b81-4667-8093-1d97b73e1488 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115652025-03-04T03:28:13.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5865b651-6b81-4667-8093-1d97b73e1488 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115662025-03-04T03:28:13.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5865b651-6b81-4667-8093-1d97b73e1488 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115672025-03-04T03:28:14.364ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a42c08d-1f26-4e53-a62d-76b81b4038b2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115682025-03-04T03:28:14.365ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a42c08d-1f26-4e53-a62d-76b81b4038b2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115692025-03-04T03:28:14.366ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2097 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6a42c08d-1f26-4e53-a62d-76b81b4038b2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115702025-03-04T03:28:15.386ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e6d9580a-579d-4d57-8652-8cebec2c9bd4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115712025-03-04T03:28:15.386ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e6d9580a-579d-4d57-8652-8cebec2c9bd4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115722025-03-04T03:28:15.388ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1872 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e6d9580a-579d-4d57-8652-8cebec2c9bd4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115732025-03-04T03:28:16.408ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9eea0ca-6fac-4fde-9afb-f1d807d304a3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115742025-03-04T03:28:16.409ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9eea0ca-6fac-4fde-9afb-f1d807d304a3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115752025-03-04T03:28:16.411ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2358 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9eea0ca-6fac-4fde-9afb-f1d807d304a3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115762025-03-04T03:28:17.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
115772025-03-04T03:28:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115782025-03-04T03:28:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115792025-03-04T03:28:17.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
115802025-03-04T03:28:17.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
115812025-03-04T03:28:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
115822025-03-04T03:28:17.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115832025-03-04T03:28:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
115842025-03-04T03:28:17.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115852025-03-04T03:28:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115862025-03-04T03:28:17.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115872025-03-04T03:28:17.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115882025-03-04T03:28:17.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115892025-03-04T03:28:17.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
115902025-03-04T03:28:17.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115912025-03-04T03:28:17.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115922025-03-04T03:28:17.227ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
115932025-03-04T03:28:17.227ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.573824ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
115942025-03-04T03:28:17.236ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
115952025-03-04T03:28:17.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
115962025-03-04T03:28:17.270ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.549459ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
115972025-03-04T03:28:17.430ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2416ce94-b265-4a9a-b288-6d416536b939 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115982025-03-04T03:28:17.430ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2416ce94-b265-4a9a-b288-6d416536b939 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
115992025-03-04T03:28:17.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1890 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2416ce94-b265-4a9a-b288-6d416536b939 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116002025-03-04T03:28:17.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 751f363d-0d2e-4a58-a55a-94aa680e7a9d roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
116012025-03-04T03:28:17.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 751f363d-0d2e-4a58-a55a-94aa680e7a9d resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
116022025-03-04T03:28:17.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2872 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 751f363d-0d2e-4a58-a55a-94aa680e7a9d response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
116032025-03-04T03:28:17.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = ef06dbc5-288b-4233-a705-15dc35a10454 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
116042025-03-04T03:28:17.894ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = ef06dbc5-288b-4233-a705-15dc35a10454 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
116052025-03-04T03:28:17.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2560 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = ef06dbc5-288b-4233-a705-15dc35a10454 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
116062025-03-04T03:28:17.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 509 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = b083588b-f419-47e8-96b8-ea22ba80a827 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
116072025-03-04T03:28:18.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be914181-96a6-43ce-9131-bef9af567ecb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116082025-03-04T03:28:18.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be914181-96a6-43ce-9131-bef9af567ecb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116092025-03-04T03:28:18.454ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2260 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = be914181-96a6-43ce-9131-bef9af567ecb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116102025-03-04T03:28:19.473ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 443a9103-65f7-497d-92eb-a9df1215d7f6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116112025-03-04T03:28:19.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 443a9103-65f7-497d-92eb-a9df1215d7f6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116122025-03-04T03:28:19.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1995 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 443a9103-65f7-497d-92eb-a9df1215d7f6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116132025-03-04T03:28:20.494ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec30f824-eea5-4454-bda8-9fad5350a613 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116142025-03-04T03:28:20.494ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec30f824-eea5-4454-bda8-9fad5350a613 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116152025-03-04T03:28:20.496ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2119 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec30f824-eea5-4454-bda8-9fad5350a613 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116162025-03-04T03:28:21.515ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 15e581f6-1c2b-4653-b0c8-4234cae583b3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116172025-03-04T03:28:21.515ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 15e581f6-1c2b-4653-b0c8-4234cae583b3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116182025-03-04T03:28:21.518ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3444 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 15e581f6-1c2b-4653-b0c8-4234cae583b3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116192025-03-04T03:28:22.544ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 384a68df-f3a2-4e26-96da-2c2b4fba5d6b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116202025-03-04T03:28:22.544ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 384a68df-f3a2-4e26-96da-2c2b4fba5d6b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116212025-03-04T03:28:22.546ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2353 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 384a68df-f3a2-4e26-96da-2c2b4fba5d6b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116222025-03-04T03:28:23.567ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fc8e9e62-e709-454c-baf4-9c4c4b32eeec roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116232025-03-04T03:28:23.567ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fc8e9e62-e709-454c-baf4-9c4c4b32eeec resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116242025-03-04T03:28:23.569ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2137 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fc8e9e62-e709-454c-baf4-9c4c4b32eeec response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116252025-03-04T03:28:24.589ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dde37fcd-d2a0-405e-af9e-0ae1455b096f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116262025-03-04T03:28:24.589ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dde37fcd-d2a0-405e-af9e-0ae1455b096f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116272025-03-04T03:28:24.591ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2233 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dde37fcd-d2a0-405e-af9e-0ae1455b096f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116282025-03-04T03:28:25.611ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e2672cad-3c81-46fb-a3d9-f4bfa0651b05 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116292025-03-04T03:28:25.611ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e2672cad-3c81-46fb-a3d9-f4bfa0651b05 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116302025-03-04T03:28:25.612ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1576 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e2672cad-3c81-46fb-a3d9-f4bfa0651b05 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116312025-03-04T03:28:26.631ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7339a31e-3684-464e-b1c8-ddd0c85e702e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116322025-03-04T03:28:26.631ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7339a31e-3684-464e-b1c8-ddd0c85e702e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116332025-03-04T03:28:26.633ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2388 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7339a31e-3684-464e-b1c8-ddd0c85e702e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116342025-03-04T03:28:27.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
116352025-03-04T03:28:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116362025-03-04T03:28:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116372025-03-04T03:28:27.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
116382025-03-04T03:28:27.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116392025-03-04T03:28:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116402025-03-04T03:28:27.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116412025-03-04T03:28:27.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
116422025-03-04T03:28:27.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116432025-03-04T03:28:27.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116442025-03-04T03:28:27.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116452025-03-04T03:28:27.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116462025-03-04T03:28:27.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116472025-03-04T03:28:27.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
116482025-03-04T03:28:27.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116492025-03-04T03:28:27.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116502025-03-04T03:28:27.232ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
116512025-03-04T03:28:27.232ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.281462ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
116522025-03-04T03:28:27.241ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116532025-03-04T03:28:27.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116542025-03-04T03:28:27.268ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.390755ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
116552025-03-04T03:28:27.652ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 811aea7a-f43f-4ba2-958e-9eb3d105047d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116562025-03-04T03:28:27.652ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 811aea7a-f43f-4ba2-958e-9eb3d105047d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116572025-03-04T03:28:27.654ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2512 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 811aea7a-f43f-4ba2-958e-9eb3d105047d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116582025-03-04T03:28:27.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 534 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 72593fee-d3bc-4b34-8ac9-dfc932f92c67 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
116592025-03-04T03:28:28.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7c05fd3-4464-43bf-b9ae-7a79f4fa769a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116602025-03-04T03:28:28.673ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7c05fd3-4464-43bf-b9ae-7a79f4fa769a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116612025-03-04T03:28:28.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1755 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f7c05fd3-4464-43bf-b9ae-7a79f4fa769a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116622025-03-04T03:28:29.694ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f783aec6-292d-4352-a1c9-b954501b48c7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116632025-03-04T03:28:29.694ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f783aec6-292d-4352-a1c9-b954501b48c7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116642025-03-04T03:28:29.696ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2091 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f783aec6-292d-4352-a1c9-b954501b48c7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116652025-03-04T03:28:30.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 424dacec-f2be-469e-bb3c-f8a88bd45622 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116662025-03-04T03:28:30.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 424dacec-f2be-469e-bb3c-f8a88bd45622 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116672025-03-04T03:28:30.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2098 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 424dacec-f2be-469e-bb3c-f8a88bd45622 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116682025-03-04T03:28:31.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61cf16b3-3274-4a72-a5fd-6e515bf967cb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116692025-03-04T03:28:31.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61cf16b3-3274-4a72-a5fd-6e515bf967cb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116702025-03-04T03:28:31.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2318 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 61cf16b3-3274-4a72-a5fd-6e515bf967cb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116712025-03-04T03:28:32.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7427e8b9-bdf3-4495-89bd-3316b13dcae0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116722025-03-04T03:28:32.761ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7427e8b9-bdf3-4495-89bd-3316b13dcae0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116732025-03-04T03:28:32.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2090 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7427e8b9-bdf3-4495-89bd-3316b13dcae0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116742025-03-04T03:28:33.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a736651d-1c28-4b4e-98aa-8dcdb81ca13c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116752025-03-04T03:28:33.783ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a736651d-1c28-4b4e-98aa-8dcdb81ca13c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116762025-03-04T03:28:33.785ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2318 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a736651d-1c28-4b4e-98aa-8dcdb81ca13c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116772025-03-04T03:28:34.804ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6664ce9b-0df7-4e5f-8186-e548eb3a9dec roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116782025-03-04T03:28:34.804ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6664ce9b-0df7-4e5f-8186-e548eb3a9dec resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116792025-03-04T03:28:34.806ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1837 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6664ce9b-0df7-4e5f-8186-e548eb3a9dec response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116802025-03-04T03:28:35.824ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f4050ef0-df16-47e7-9454-333e68fbcfbf roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116812025-03-04T03:28:35.824ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f4050ef0-df16-47e7-9454-333e68fbcfbf resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116822025-03-04T03:28:35.826ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2166 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f4050ef0-df16-47e7-9454-333e68fbcfbf response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116832025-03-04T03:28:36.847ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bec7c6ba-f20a-4220-93fc-8efdbac60de3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116842025-03-04T03:28:36.847ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bec7c6ba-f20a-4220-93fc-8efdbac60de3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116852025-03-04T03:28:36.849ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2054 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bec7c6ba-f20a-4220-93fc-8efdbac60de3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
116862025-03-04T03:28:37.204ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
116872025-03-04T03:28:37.204ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
116882025-03-04T03:28:37.204ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
116892025-03-04T03:28:37.204ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
116902025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
116912025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
116922025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
116932025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
116942025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
116952025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
116962025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
116972025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
116982025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
116992025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
117002025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
117012025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
117022025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
117032025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
117042025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
117052025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117062025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
117072025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117082025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
117092025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
117102025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
117112025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
117122025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
117132025-03-04T03:28:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
117142025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
117152025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
117162025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
117172025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
117182025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
117192025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
117202025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
117212025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
117222025-03-04T03:28:37.206ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
117232025-03-04T03:28:37.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
117242025-03-04T03:28:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117252025-03-04T03:28:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
117262025-03-04T03:28:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
117272025-03-04T03:28:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
117282025-03-04T03:28:37.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1629, tv_nsec: 402314145 } })
117292025-03-04T03:28:37.207ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117302025-03-04T03:28:37.207ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1629, tv_nsec: 402461437 } })
117312025-03-04T03:28:37.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
117322025-03-04T03:28:37.207ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
117332025-03-04T03:28:37.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117342025-03-04T03:28:37.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117352025-03-04T03:28:37.208ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117362025-03-04T03:28:37.209ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.785026ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
117372025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
117382025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117392025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
117402025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
117412025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
117422025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
117432025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
117442025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
117452025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
117462025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
117472025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
117482025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
117492025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117502025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
117512025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
117522025-03-04T03:28:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
117532025-03-04T03:28:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
117542025-03-04T03:28:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
117552025-03-04T03:28:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
117562025-03-04T03:28:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
117572025-03-04T03:28:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
117582025-03-04T03:28:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
117592025-03-04T03:28:37.211ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
117602025-03-04T03:28:37.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
117612025-03-04T03:28:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
117622025-03-04T03:28:37.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117632025-03-04T03:28:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
117642025-03-04T03:28:37.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117652025-03-04T03:28:37.214ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117662025-03-04T03:28:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
117672025-03-04T03:28:37.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
117682025-03-04T03:28:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
117692025-03-04T03:28:37.214ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117702025-03-04T03:28:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
117712025-03-04T03:28:37.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
117722025-03-04T03:28:37.216ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117732025-03-04T03:28:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
117742025-03-04T03:28:37.216ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
117752025-03-04T03:28:37.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
117762025-03-04T03:28:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117772025-03-04T03:28:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
117782025-03-04T03:28:37.220ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1629, tv_nsec: 415018961 } })
117792025-03-04T03:28:37.220ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
117802025-03-04T03:28:37.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117812025-03-04T03:28:37.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
117822025-03-04T03:28:37.221ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
117832025-03-04T03:28:37.222ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117842025-03-04T03:28:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
117852025-03-04T03:28:37.223ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
117862025-03-04T03:28:37.224ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
117872025-03-04T03:28:37.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
117882025-03-04T03:28:37.254ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117892025-03-04T03:28:37.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117902025-03-04T03:28:37.257ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117912025-03-04T03:28:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117922025-03-04T03:28:37.263ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
117932025-03-04T03:28:37.263ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
117942025-03-04T03:28:37.263ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
117952025-03-04T03:28:37.263ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
117962025-03-04T03:28:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117972025-03-04T03:28:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
117982025-03-04T03:28:37.264ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
117992025-03-04T03:28:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
118002025-03-04T03:28:37.265ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 54.864982ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
118012025-03-04T03:28:37.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 55.670852ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
118022025-03-04T03:28:37.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 61.243461ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
118032025-03-04T03:28:37.267ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
118042025-03-04T03:28:37.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
118052025-03-04T03:28:37.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118062025-03-04T03:28:37.268ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
118072025-03-04T03:28:37.268ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
118082025-03-04T03:28:37.268ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "03262215-e667-42b7-8e62-9c00e15cf197", "content-length": "26", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
118092025-03-04T03:28:37.268ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
118102025-03-04T03:28:37.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
118112025-03-04T03:28:37.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
118122025-03-04T03:28:37.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
118132025-03-04T03:28:37.270ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
118142025-03-04T03:28:37.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 62.014171ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
118152025-03-04T03:28:37.273ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
118162025-03-04T03:28:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
118172025-03-04T03:28:37.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118182025-03-04T03:28:37.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
118192025-03-04T03:28:37.279ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
118202025-03-04T03:28:37.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
118212025-03-04T03:28:37.280ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
118222025-03-04T03:28:37.280ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
118232025-03-04T03:28:37.281ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "09316968-71b8-42d7-92a4-974b5fb9ba61", "content-length": "26", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
118242025-03-04T03:28:37.281ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
118252025-03-04T03:28:37.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
118262025-03-04T03:28:37.282ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
118272025-03-04T03:28:37.282ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 76.018392ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
118282025-03-04T03:28:37.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
118292025-03-04T03:28:37.283ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
118302025-03-04T03:28:37.284ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118312025-03-04T03:28:37.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
118322025-03-04T03:28:37.286ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
118332025-03-04T03:28:37.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
118342025-03-04T03:28:37.290ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 85.080113ms iteration = 25 status = {"physical_disks_added":0}
118352025-03-04T03:28:37.292ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
118362025-03-04T03:28:37.292ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
118372025-03-04T03:28:37.292ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1771dc9-bc2f-4583-a754-76e133212273", "content-length": "26", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
118382025-03-04T03:28:37.292ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
118392025-03-04T03:28:37.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118402025-03-04T03:28:37.293ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
118412025-03-04T03:28:37.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 88.12466ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
118422025-03-04T03:28:37.294ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
118432025-03-04T03:28:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
118442025-03-04T03:28:37.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
118452025-03-04T03:28:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
118462025-03-04T03:28:37.297ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 92.364112ms iteration = 22 status = 0
118472025-03-04T03:28:37.305ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
118482025-03-04T03:28:37.305ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
118492025-03-04T03:28:37.305ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
118502025-03-04T03:28:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
118512025-03-04T03:28:37.307ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 100.226178ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
118522025-03-04T03:28:37.309ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
118532025-03-04T03:28:37.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
118542025-03-04T03:28:37.314ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
118552025-03-04T03:28:37.314ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 109.480952ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
118562025-03-04T03:28:37.317ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 107.124583ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
118572025-03-04T03:28:37.335ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
118582025-03-04T03:28:37.335ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
118592025-03-04T03:28:37.335ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 128.826969ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
118602025-03-04T03:28:37.339ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
118612025-03-04T03:28:37.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
118622025-03-04T03:28:37.478ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
118632025-03-04T03:28:37.478ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b255d573-e83c-44c4-b10c-f5db6898432f", "content-length": "1", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
118642025-03-04T03:28:37.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
118652025-03-04T03:28:37.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
118662025-03-04T03:28:37.502ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 298.205955ms iteration = 22 status = {}
118672025-03-04T03:28:37.524ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
118682025-03-04T03:28:37.524ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
118692025-03-04T03:28:37.524ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "84c86da4-1d91-41b2-addd-6af752115aaa", "content-length": "372", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
118702025-03-04T03:28:37.524ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118712025-03-04T03:28:37.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118722025-03-04T03:28:37.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 323.153681ms iteration = 22 status = null
118732025-03-04T03:28:37.556ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 345.760947ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
118742025-03-04T03:28:37.563ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
118752025-03-04T03:28:37.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
118762025-03-04T03:28:37.575ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
118772025-03-04T03:28:37.575ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
118782025-03-04T03:28:37.575ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e47d92f6-1c55-48fc-9a5c-a1656e17e8b9", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
118792025-03-04T03:28:37.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118802025-03-04T03:28:37.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118812025-03-04T03:28:37.577ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118822025-03-04T03:28:37.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118832025-03-04T03:28:37.581ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118842025-03-04T03:28:37.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118852025-03-04T03:28:37.584ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
118862025-03-04T03:28:37.584ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
118872025-03-04T03:28:37.585ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "527891bd-cc62-4ff6-aa02-9f51623a87f9", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
118882025-03-04T03:28:37.586ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118892025-03-04T03:28:37.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118902025-03-04T03:28:37.587ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 380.096489ms iteration = 22 status = {}
118912025-03-04T03:28:37.588ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
118922025-03-04T03:28:37.588ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118932025-03-04T03:28:37.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118942025-03-04T03:28:37.593ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118952025-03-04T03:28:37.595ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
118962025-03-04T03:28:37.595ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
118972025-03-04T03:28:37.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
118982025-03-04T03:28:37.598ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
118992025-03-04T03:28:37.598ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d3201c5-6fda-45ce-90f5-eb7c225ca4b2", "content-length": "134", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
119002025-03-04T03:28:37.598ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
119012025-03-04T03:28:37.598ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
119022025-03-04T03:28:37.598ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
119032025-03-04T03:28:37.598ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
119042025-03-04T03:28:37.598ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
119052025-03-04T03:28:37.598ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
119062025-03-04T03:28:37.598ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
119072025-03-04T03:28:37.598ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
119082025-03-04T03:28:37.599ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
119092025-03-04T03:28:37.599ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
119102025-03-04T03:28:37.599ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
119112025-03-04T03:28:37.599ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119122025-03-04T03:28:37.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119132025-03-04T03:28:37.601ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
119142025-03-04T03:28:37.601ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "48dbc78f-b1e3-4fee-ada1-e066571e1d6f", "content-length": "2", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
119152025-03-04T03:28:37.601ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
119162025-03-04T03:28:37.601ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "fadabc7f-cf98-4e5f-88fc-b35acafb43b2", "content-length": "51", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
119172025-03-04T03:28:37.601ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
119182025-03-04T03:28:37.601ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
119192025-03-04T03:28:37.624ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119202025-03-04T03:28:37.626ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
119212025-03-04T03:28:37.626ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
119222025-03-04T03:28:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
119232025-03-04T03:28:37.702ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b3339e0f-6b69-42b7-8a0c-7bfa1a361dd5", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
119242025-03-04T03:28:37.702ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
119252025-03-04T03:28:37.702ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
119262025-03-04T03:28:37.702ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5b6baa8-dadd-492b-9954-3fca006657e2", "content-length": "682", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
119272025-03-04T03:28:37.702ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119282025-03-04T03:28:37.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119292025-03-04T03:28:37.706ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119302025-03-04T03:28:37.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119312025-03-04T03:28:37.710ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119322025-03-04T03:28:37.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119332025-03-04T03:28:37.713ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119342025-03-04T03:28:37.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119352025-03-04T03:28:37.715ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
119362025-03-04T03:28:37.715ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f9bd7cb-72b9-412f-9086-9b1802b7f69d", "content-length": "691", "date": "Tue, 04 Mar 2025 03:28:37 GMT"} })
119372025-03-04T03:28:37.715ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
119382025-03-04T03:28:37.715ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
119392025-03-04T03:28:37.715ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
119402025-03-04T03:28:37.717ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 511.768492ms iteration = 22 status = {}
119412025-03-04T03:28:37.721ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 515.838773ms iteration = 22 status = {}
119422025-03-04T03:28:37.869ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 237c5690-a3b0-43ba-9523-6aa76b9b6830 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119432025-03-04T03:28:37.870ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 237c5690-a3b0-43ba-9523-6aa76b9b6830 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119442025-03-04T03:28:37.872ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2842 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 237c5690-a3b0-43ba-9523-6aa76b9b6830 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119452025-03-04T03:28:37.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 461 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 1730e4bd-6846-48fc-a25d-eb956fff9748 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
119462025-03-04T03:28:38.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e763c8be-6b23-4614-ae14-b0cabedcf2b6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119472025-03-04T03:28:38.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e763c8be-6b23-4614-ae14-b0cabedcf2b6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119482025-03-04T03:28:38.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2444 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e763c8be-6b23-4614-ae14-b0cabedcf2b6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119492025-03-04T03:28:39.912ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5b05c225-f4c0-461a-b69f-1d2bc143ece9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119502025-03-04T03:28:39.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5b05c225-f4c0-461a-b69f-1d2bc143ece9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119512025-03-04T03:28:39.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2378 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5b05c225-f4c0-461a-b69f-1d2bc143ece9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119522025-03-04T03:28:40.934ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5782de78-84f4-4b70-8d4d-6c4aaa99ec4a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119532025-03-04T03:28:40.934ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5782de78-84f4-4b70-8d4d-6c4aaa99ec4a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119542025-03-04T03:28:40.936ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2479 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5782de78-84f4-4b70-8d4d-6c4aaa99ec4a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119552025-03-04T03:28:41.955ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3b8af133-0c6c-46ed-bbee-4e38aa561dff roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119562025-03-04T03:28:41.955ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3b8af133-0c6c-46ed-bbee-4e38aa561dff resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119572025-03-04T03:28:41.958ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2858 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3b8af133-0c6c-46ed-bbee-4e38aa561dff response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119582025-03-04T03:28:42.978ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1c2e8ce3-b99d-4600-9f9a-2e4b67ddee19 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119592025-03-04T03:28:42.978ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1c2e8ce3-b99d-4600-9f9a-2e4b67ddee19 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119602025-03-04T03:28:42.980ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2375 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1c2e8ce3-b99d-4600-9f9a-2e4b67ddee19 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119612025-03-04T03:28:44.001ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 308b3c7e-3c35-4baf-9174-81d4b823c1cc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119622025-03-04T03:28:44.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 308b3c7e-3c35-4baf-9174-81d4b823c1cc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119632025-03-04T03:28:44.003ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2145 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 308b3c7e-3c35-4baf-9174-81d4b823c1cc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119642025-03-04T03:28:45.025ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb69158a-5c21-4804-91b0-83d22035ada4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119652025-03-04T03:28:45.025ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb69158a-5c21-4804-91b0-83d22035ada4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119662025-03-04T03:28:45.027ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2429 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fb69158a-5c21-4804-91b0-83d22035ada4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119672025-03-04T03:28:46.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cbf4d67b-1f2b-4e21-b2ad-c44fc3808d4a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119682025-03-04T03:28:46.047ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cbf4d67b-1f2b-4e21-b2ad-c44fc3808d4a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119692025-03-04T03:28:46.049ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2604 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cbf4d67b-1f2b-4e21-b2ad-c44fc3808d4a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119702025-03-04T03:28:47.068ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6ba42ee-6d61-4f44-b828-aa3fa6c8cd19 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119712025-03-04T03:28:47.068ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6ba42ee-6d61-4f44-b828-aa3fa6c8cd19 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119722025-03-04T03:28:47.070ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2522 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6ba42ee-6d61-4f44-b828-aa3fa6c8cd19 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
119732025-03-04T03:28:47.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
119742025-03-04T03:28:47.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119752025-03-04T03:28:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119762025-03-04T03:28:47.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
119772025-03-04T03:28:47.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119782025-03-04T03:28:47.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
119792025-03-04T03:28:47.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119802025-03-04T03:28:47.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
119812025-03-04T03:28:47.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119822025-03-04T03:28:47.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119832025-03-04T03:28:47.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119842025-03-04T03:28:47.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
119852025-03-04T03:28:47.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119862025-03-04T03:28:47.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119872025-03-04T03:28:47.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.708862ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
119882025-03-04T03:28:47.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119892025-03-04T03:28:47.272ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
119902025-03-04T03:28:47.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119912025-03-04T03:28:47.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119922025-03-04T03:28:47.285ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
119932025-03-04T03:28:47.286ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 80.441016ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
119942025-03-04T03:28:47.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 1d62a792-3209-49d0-afee-773ccd9cb527 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
119952025-03-04T03:28:47.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 1d62a792-3209-49d0-afee-773ccd9cb527 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
119962025-03-04T03:28:47.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3293 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 1d62a792-3209-49d0-afee-773ccd9cb527 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
119972025-03-04T03:28:47.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = d09db691-2092-4bf0-a633-b0f9a2f13003 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
119982025-03-04T03:28:47.894ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = d09db691-2092-4bf0-a633-b0f9a2f13003 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
119992025-03-04T03:28:47.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1985 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = d09db691-2092-4bf0-a633-b0f9a2f13003 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
120002025-03-04T03:28:47.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 468 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 38079454-9254-45b5-9155-ebc568940411 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
120012025-03-04T03:28:48.090ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 78230f07-50e3-4e2d-859d-de82a845f9cc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120022025-03-04T03:28:48.090ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 78230f07-50e3-4e2d-859d-de82a845f9cc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120032025-03-04T03:28:48.092ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2177 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 78230f07-50e3-4e2d-859d-de82a845f9cc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120042025-03-04T03:28:49.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd6896f8-e297-4818-8d04-556f3e4b080a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120052025-03-04T03:28:49.111ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd6896f8-e297-4818-8d04-556f3e4b080a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120062025-03-04T03:28:49.112ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2140 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd6896f8-e297-4818-8d04-556f3e4b080a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120072025-03-04T03:28:50.132ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b01280ef-c567-49e9-9669-e48f1a121bc3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120082025-03-04T03:28:50.132ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b01280ef-c567-49e9-9669-e48f1a121bc3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120092025-03-04T03:28:50.134ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1957 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b01280ef-c567-49e9-9669-e48f1a121bc3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120102025-03-04T03:28:51.153ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aac3d29f-8f17-4445-a3e6-43f9809c43ed roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120112025-03-04T03:28:51.153ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aac3d29f-8f17-4445-a3e6-43f9809c43ed resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120122025-03-04T03:28:51.155ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1786 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aac3d29f-8f17-4445-a3e6-43f9809c43ed response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120132025-03-04T03:28:52.174ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6d38385-7333-4475-915f-121609b3a9cc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120142025-03-04T03:28:52.174ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6d38385-7333-4475-915f-121609b3a9cc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120152025-03-04T03:28:52.176ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2222 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a6d38385-7333-4475-915f-121609b3a9cc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120162025-03-04T03:28:53.195ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c7abbd5b-f96a-483a-8073-410469d90d6c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120172025-03-04T03:28:53.195ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c7abbd5b-f96a-483a-8073-410469d90d6c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120182025-03-04T03:28:53.197ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2076 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c7abbd5b-f96a-483a-8073-410469d90d6c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120192025-03-04T03:28:54.217ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e07bb320-bdbf-47b1-98d4-f6065193b834 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120202025-03-04T03:28:54.218ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e07bb320-bdbf-47b1-98d4-f6065193b834 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120212025-03-04T03:28:54.220ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2161 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e07bb320-bdbf-47b1-98d4-f6065193b834 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120222025-03-04T03:28:55.238ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ecf77b8d-ebd1-46d1-86f3-eed9d70939a7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120232025-03-04T03:28:55.238ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ecf77b8d-ebd1-46d1-86f3-eed9d70939a7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120242025-03-04T03:28:55.240ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2287 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ecf77b8d-ebd1-46d1-86f3-eed9d70939a7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120252025-03-04T03:28:56.261ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0462baa0-b35d-4f9b-8f38-53bee245a721 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120262025-03-04T03:28:56.261ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0462baa0-b35d-4f9b-8f38-53bee245a721 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120272025-03-04T03:28:56.262ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1913 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0462baa0-b35d-4f9b-8f38-53bee245a721 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120282025-03-04T03:28:57.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
120292025-03-04T03:28:57.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120302025-03-04T03:28:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120312025-03-04T03:28:57.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
120322025-03-04T03:28:57.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120332025-03-04T03:28:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120342025-03-04T03:28:57.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120352025-03-04T03:28:57.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120362025-03-04T03:28:57.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120372025-03-04T03:28:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
120382025-03-04T03:28:57.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120392025-03-04T03:28:57.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120402025-03-04T03:28:57.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120412025-03-04T03:28:57.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120422025-03-04T03:28:57.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120432025-03-04T03:28:57.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
120442025-03-04T03:28:57.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120452025-03-04T03:28:57.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120462025-03-04T03:28:57.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
120472025-03-04T03:28:57.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.596103ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
120482025-03-04T03:28:57.238ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.752179ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
120492025-03-04T03:28:57.281ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af159456-09b6-4401-aecc-f13faf8ddc5e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120502025-03-04T03:28:57.281ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af159456-09b6-4401-aecc-f13faf8ddc5e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120512025-03-04T03:28:57.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2562 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af159456-09b6-4401-aecc-f13faf8ddc5e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120522025-03-04T03:28:57.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 570 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = b14db5f0-e465-4689-b1d3-71b0a6ba8d05 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
120532025-03-04T03:28:58.302ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd9eb675-91c7-409a-bc88-93315b4add39 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120542025-03-04T03:28:58.303ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd9eb675-91c7-409a-bc88-93315b4add39 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120552025-03-04T03:28:58.304ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1989 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cd9eb675-91c7-409a-bc88-93315b4add39 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120562025-03-04T03:28:59.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0c891c8e-ff07-4456-abe0-2bad55eb21c2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120572025-03-04T03:28:59.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0c891c8e-ff07-4456-abe0-2bad55eb21c2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120582025-03-04T03:28:59.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2324 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0c891c8e-ff07-4456-abe0-2bad55eb21c2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120592025-03-04T03:29:00.342ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d912207-eedb-4900-9936-049cd547fa03 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120602025-03-04T03:29:00.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d912207-eedb-4900-9936-049cd547fa03 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120612025-03-04T03:29:00.344ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1798 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d912207-eedb-4900-9936-049cd547fa03 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120622025-03-04T03:29:01.363ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 238db71b-3ce5-4934-8483-31b3a55811c4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120632025-03-04T03:29:01.363ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 238db71b-3ce5-4934-8483-31b3a55811c4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120642025-03-04T03:29:01.366ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2374 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 238db71b-3ce5-4934-8483-31b3a55811c4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120652025-03-04T03:29:02.384ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eb9f7d33-ab61-4ac9-9985-c45897f6df8d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120662025-03-04T03:29:02.385ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eb9f7d33-ab61-4ac9-9985-c45897f6df8d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120672025-03-04T03:29:02.386ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1914 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eb9f7d33-ab61-4ac9-9985-c45897f6df8d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120682025-03-04T03:29:02.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 056f34a9-7bf2-4b64-8cec-b4f66a04a536 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
120692025-03-04T03:29:02.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 056f34a9-7bf2-4b64-8cec-b4f66a04a536 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
120702025-03-04T03:29:02.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4844 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 056f34a9-7bf2-4b64-8cec-b4f66a04a536 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
120712025-03-04T03:29:02.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 399fc7fa-6c8b-4d2e-902c-5eb7f1e82e96 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
120722025-03-04T03:29:02.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 399fc7fa-6c8b-4d2e-902c-5eb7f1e82e96 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
120732025-03-04T03:29:02.900ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3314 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 399fc7fa-6c8b-4d2e-902c-5eb7f1e82e96 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
120742025-03-04T03:29:03.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a11d78e6-0eff-4cd0-9107-bcbf4d622907 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120752025-03-04T03:29:03.405ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a11d78e6-0eff-4cd0-9107-bcbf4d622907 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120762025-03-04T03:29:03.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1958 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a11d78e6-0eff-4cd0-9107-bcbf4d622907 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120772025-03-04T03:29:04.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0fc7679-9b22-47ef-b1a1-7993c37e66e9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120782025-03-04T03:29:04.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0fc7679-9b22-47ef-b1a1-7993c37e66e9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120792025-03-04T03:29:04.430ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2527 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0fc7679-9b22-47ef-b1a1-7993c37e66e9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120802025-03-04T03:29:05.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3af7c83b-d05d-45a3-8854-f6a99be8b43a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120812025-03-04T03:29:05.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3af7c83b-d05d-45a3-8854-f6a99be8b43a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120822025-03-04T03:29:05.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2007 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3af7c83b-d05d-45a3-8854-f6a99be8b43a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120832025-03-04T03:29:06.473ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 54d1c8d2-e347-4f97-ba04-6b83d6fd845b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120842025-03-04T03:29:06.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 54d1c8d2-e347-4f97-ba04-6b83d6fd845b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120852025-03-04T03:29:06.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2030 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 54d1c8d2-e347-4f97-ba04-6b83d6fd845b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
120862025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
120872025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
120882025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
120892025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
120902025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
120912025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
120922025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
120932025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
120942025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
120952025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
120962025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
120972025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
120982025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
120992025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
121002025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
121012025-03-04T03:29:07.204ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:19:07.204953852 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
121022025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
121032025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
121042025-03-04T03:29:07.204ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
121052025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
121062025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
121072025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
121082025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
121092025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
121102025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
121112025-03-04T03:29:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1659, tv_nsec: 400310395 } })
121122025-03-04T03:29:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1659, tv_nsec: 400445116 } })
121132025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
121142025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
121152025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
121162025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
121172025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
121182025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
121192025-03-04T03:29:07.205ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
121202025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121212025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.98\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
121222025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
121232025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
121242025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
121252025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
121262025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
121272025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
121282025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121292025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
121302025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
121312025-03-04T03:29:07.205ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
121322025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
121332025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
121342025-03-04T03:29:07.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
121352025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121362025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
121372025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
121382025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
121392025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.187774ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
121402025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121412025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
121422025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
121432025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.183745ms iteration = 12 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
121442025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121452025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
121462025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
121472025-03-04T03:29:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
121482025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
121492025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
121502025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
121512025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121522025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
121532025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
121542025-03-04T03:29:07.207ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
121552025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121562025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
121572025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
121582025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
121592025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
121602025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121612025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
121622025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121632025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121642025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
121652025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
121662025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
121672025-03-04T03:29:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
121682025-03-04T03:29:07.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1659, tv_nsec: 402810145 } })
121692025-03-04T03:29:07.208ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
121702025-03-04T03:29:07.208ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:19:07.204953852 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
121712025-03-04T03:29:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.558814ms iteration = 12 status = {"expiration":"2025-03-04T03:19:07.204953852Z","pruned":[]}
121722025-03-04T03:29:07.208ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
121732025-03-04T03:29:07.208ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
121742025-03-04T03:29:07.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
121752025-03-04T03:29:07.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
121762025-03-04T03:29:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
121772025-03-04T03:29:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
121782025-03-04T03:29:07.209ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1659, tv_nsec: 404043821 } })
121792025-03-04T03:29:07.209ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
121802025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121812025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
121822025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
121832025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
121842025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
121852025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
121862025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
121872025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
121882025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121892025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
121902025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
121912025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
121922025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
121932025-03-04T03:29:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
121942025-03-04T03:29:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
121952025-03-04T03:29:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121962025-03-04T03:29:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
121972025-03-04T03:29:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
121982025-03-04T03:29:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
121992025-03-04T03:29:07.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122002025-03-04T03:29:07.212ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122012025-03-04T03:29:07.212ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.962176ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
122022025-03-04T03:29:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
122032025-03-04T03:29:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122042025-03-04T03:29:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122052025-03-04T03:29:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
122062025-03-04T03:29:07.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
122072025-03-04T03:29:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
122082025-03-04T03:29:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
122092025-03-04T03:29:07.214ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122102025-03-04T03:29:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122112025-03-04T03:29:07.215ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122122025-03-04T03:29:07.216ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122132025-03-04T03:29:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
122142025-03-04T03:29:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
122152025-03-04T03:29:07.219ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
122162025-03-04T03:29:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
122172025-03-04T03:29:07.220ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122182025-03-04T03:29:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
122192025-03-04T03:29:07.223ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
122202025-03-04T03:29:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
122212025-03-04T03:29:07.225ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122222025-03-04T03:29:07.227ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
122232025-03-04T03:29:07.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
122242025-03-04T03:29:07.228ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
122252025-03-04T03:29:07.229ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.503497ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
122262025-03-04T03:29:07.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122272025-03-04T03:29:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122282025-03-04T03:29:07.239ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
122292025-03-04T03:29:07.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
122302025-03-04T03:29:07.243ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
122312025-03-04T03:29:07.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
122322025-03-04T03:29:07.245ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1659, tv_nsec: 439821249 } })
122332025-03-04T03:29:07.245ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
122342025-03-04T03:29:07.262ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122352025-03-04T03:29:07.263ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122362025-03-04T03:29:07.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
122372025-03-04T03:29:07.265ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
122382025-03-04T03:29:07.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
122392025-03-04T03:29:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
122402025-03-04T03:29:07.266ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
122412025-03-04T03:29:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
122422025-03-04T03:29:07.267ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 56.37087ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
122432025-03-04T03:29:07.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122442025-03-04T03:29:07.269ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
122452025-03-04T03:29:07.269ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
122462025-03-04T03:29:07.269ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
122472025-03-04T03:29:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
122482025-03-04T03:29:07.271ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
122492025-03-04T03:29:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
122502025-03-04T03:29:07.272ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122512025-03-04T03:29:07.272ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
122522025-03-04T03:29:07.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122532025-03-04T03:29:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122542025-03-04T03:29:07.273ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
122552025-03-04T03:29:07.274ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122562025-03-04T03:29:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122572025-03-04T03:29:07.275ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
122582025-03-04T03:29:07.275ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
122592025-03-04T03:29:07.275ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
122602025-03-04T03:29:07.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
122612025-03-04T03:29:07.275ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122622025-03-04T03:29:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
122632025-03-04T03:29:07.276ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
122642025-03-04T03:29:07.276ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
122652025-03-04T03:29:07.276ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 68.158385ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
122662025-03-04T03:29:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
122672025-03-04T03:29:07.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
122682025-03-04T03:29:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
122692025-03-04T03:29:07.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122702025-03-04T03:29:07.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
122712025-03-04T03:29:07.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
122722025-03-04T03:29:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
122732025-03-04T03:29:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
122742025-03-04T03:29:07.281ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
122752025-03-04T03:29:07.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122762025-03-04T03:29:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122772025-03-04T03:29:07.282ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 71.290884ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
122782025-03-04T03:29:07.283ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
122792025-03-04T03:29:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
122802025-03-04T03:29:07.287ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
122812025-03-04T03:29:07.287ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
122822025-03-04T03:29:07.287ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
122832025-03-04T03:29:07.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
122842025-03-04T03:29:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
122852025-03-04T03:29:07.289ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122862025-03-04T03:29:07.292ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
122872025-03-04T03:29:07.292ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 86.45966ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
122882025-03-04T03:29:07.293ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 82.291729ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
122892025-03-04T03:29:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
122902025-03-04T03:29:07.293ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
122912025-03-04T03:29:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
122922025-03-04T03:29:07.298ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
122932025-03-04T03:29:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
122942025-03-04T03:29:07.305ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
122952025-03-04T03:29:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
122962025-03-04T03:29:07.307ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122972025-03-04T03:29:07.308ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
122982025-03-04T03:29:07.308ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 101.244741ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
122992025-03-04T03:29:07.309ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123002025-03-04T03:29:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
123012025-03-04T03:29:07.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
123022025-03-04T03:29:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
123032025-03-04T03:29:07.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
123042025-03-04T03:29:07.313ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
123052025-03-04T03:29:07.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
123062025-03-04T03:29:07.314ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
123072025-03-04T03:29:07.314ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
123082025-03-04T03:29:07.314ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 108.298107ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
123092025-03-04T03:29:07.315ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 109.866486ms iteration = 23 status = 0
123102025-03-04T03:29:07.316ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
123112025-03-04T03:29:07.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
123122025-03-04T03:29:07.318ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123132025-03-04T03:29:07.318ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
123142025-03-04T03:29:07.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
123152025-03-04T03:29:07.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
123162025-03-04T03:29:07.319ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
123172025-03-04T03:29:07.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
123182025-03-04T03:29:07.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
123192025-03-04T03:29:07.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
123202025-03-04T03:29:07.321ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123212025-03-04T03:29:07.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
123222025-03-04T03:29:07.323ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
123232025-03-04T03:29:07.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
123242025-03-04T03:29:07.326ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 115.682778ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
123252025-03-04T03:29:07.328ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
123262025-03-04T03:29:07.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
123272025-03-04T03:29:07.328ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 123.262361ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
123282025-03-04T03:29:07.331ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 124.532917ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
123292025-03-04T03:29:07.331ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123302025-03-04T03:29:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
123312025-03-04T03:29:07.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
123322025-03-04T03:29:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
123332025-03-04T03:29:07.338ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
123342025-03-04T03:29:07.338ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
123352025-03-04T03:29:07.338ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
123362025-03-04T03:29:07.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
123372025-03-04T03:29:07.339ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123382025-03-04T03:29:07.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
123392025-03-04T03:29:07.342ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
123402025-03-04T03:29:07.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
123412025-03-04T03:29:07.344ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
123422025-03-04T03:29:07.344ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
123432025-03-04T03:29:07.344ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
123442025-03-04T03:29:07.344ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 139.928685ms iteration = 12 status = {"generation":2}
123452025-03-04T03:29:07.344ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
123462025-03-04T03:29:07.352ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123472025-03-04T03:29:07.354ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
123482025-03-04T03:29:07.354ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
123492025-03-04T03:29:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
123502025-03-04T03:29:07.355ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
123512025-03-04T03:29:07.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
123522025-03-04T03:29:07.359ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123532025-03-04T03:29:07.359ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
123542025-03-04T03:29:07.359ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 152.533969ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
123552025-03-04T03:29:07.364ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
123562025-03-04T03:29:07.364ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 158.607674ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
123572025-03-04T03:29:07.368ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 157.308548ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
123582025-03-04T03:29:07.384ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
123592025-03-04T03:29:07.384ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
123602025-03-04T03:29:07.384ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
123612025-03-04T03:29:07.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
123622025-03-04T03:29:07.386ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123632025-03-04T03:29:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
123642025-03-04T03:29:07.389ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
123652025-03-04T03:29:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
123662025-03-04T03:29:07.392ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
123672025-03-04T03:29:07.392ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
123682025-03-04T03:29:07.392ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
123692025-03-04T03:29:07.392ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 187.209474ms iteration = 12 status = {"generation":1}
123702025-03-04T03:29:07.400ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 195.182681ms iteration = 26 status = {"physical_disks_added":0}
123712025-03-04T03:29:07.405ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
123722025-03-04T03:29:07.405ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 200.918123ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
123732025-03-04T03:29:07.427ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
123742025-03-04T03:29:07.427ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 220.679345ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
123752025-03-04T03:29:07.464ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
123762025-03-04T03:29:07.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
123772025-03-04T03:29:07.466ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123782025-03-04T03:29:07.468ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
123792025-03-04T03:29:07.468ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
123802025-03-04T03:29:07.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
123812025-03-04T03:29:07.469ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123822025-03-04T03:29:07.471ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
123832025-03-04T03:29:07.471ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
123842025-03-04T03:29:07.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
123852025-03-04T03:29:07.474ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
123862025-03-04T03:29:07.474ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
123872025-03-04T03:29:07.474ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "427581a6-3461-4f2a-b3c1-621266bf0806", "content-length": "26", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
123882025-03-04T03:29:07.475ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
123892025-03-04T03:29:07.477ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
123902025-03-04T03:29:07.482ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
123912025-03-04T03:29:07.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d4ea187-594c-4858-95e3-e694cbafd152 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
123922025-03-04T03:29:07.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d4ea187-594c-4858-95e3-e694cbafd152 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
123932025-03-04T03:29:07.502ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
123942025-03-04T03:29:07.502ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
123952025-03-04T03:29:07.503ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "889957c3-937f-4bf1-a29f-db8d3ca5253e", "content-length": "26", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
123962025-03-04T03:29:07.503ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
123972025-03-04T03:29:07.504ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
123982025-03-04T03:29:07.547ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
123992025-03-04T03:29:07.605ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
124002025-03-04T03:29:07.605ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
124012025-03-04T03:29:07.605ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9f02688-dd71-42ce-9c31-c4d6d2e620b4", "content-length": "26", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124022025-03-04T03:29:07.605ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
124032025-03-04T03:29:07.606ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
124042025-03-04T03:29:07.696ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
124052025-03-04T03:29:07.696ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "1eb41278-50bc-4863-88cf-91964c210840", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
124062025-03-04T03:29:07.696ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
124072025-03-04T03:29:07.698ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "58436252-ba74-43db-9d3b-240fc05d87eb", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
124082025-03-04T03:29:07.698ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
124092025-03-04T03:29:07.699ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
124102025-03-04T03:29:07.699ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 494.957916ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
124112025-03-04T03:29:07.760ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
124122025-03-04T03:29:07.760ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
124132025-03-04T03:29:07.760ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "dafcc191-544f-4713-9be3-10318aa36013", "content-length": "372", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124142025-03-04T03:29:07.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124152025-03-04T03:29:07.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124162025-03-04T03:29:07.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 267623 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d4ea187-594c-4858-95e3-e694cbafd152 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
124172025-03-04T03:29:07.794ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
124182025-03-04T03:29:07.795ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba4e4134-eab0-4dc1-ab8d-a61f6797d39d", "content-length": "1", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124192025-03-04T03:29:07.797ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
124202025-03-04T03:29:07.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
124212025-03-04T03:29:07.799ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 594.728469ms iteration = 23 status = null
124222025-03-04T03:29:07.801ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 596.091895ms iteration = 23 status = {}
124232025-03-04T03:29:07.805ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
124242025-03-04T03:29:07.805ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
124252025-03-04T03:29:07.805ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ff2ad9a0-13f5-4f45-89d6-f5b2e1b797bf", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124262025-03-04T03:29:07.805ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124272025-03-04T03:29:07.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124282025-03-04T03:29:07.808ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124292025-03-04T03:29:07.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124302025-03-04T03:29:07.808ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
124312025-03-04T03:29:07.809ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d9af3d4a-336b-4f1f-9f53-260a73cb6811", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
124322025-03-04T03:29:07.809ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
124332025-03-04T03:29:07.810ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "81f8919a-849d-474c-bc25-b1baf4d493b2", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
124342025-03-04T03:29:07.810ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
124352025-03-04T03:29:07.811ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124362025-03-04T03:29:07.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124372025-03-04T03:29:07.812ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "df6b5746-3582-4890-beb0-06bdf9a4fb37", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
124382025-03-04T03:29:07.812ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
124392025-03-04T03:29:07.814ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
124402025-03-04T03:29:07.814ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
124412025-03-04T03:29:07.814ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
124422025-03-04T03:29:07.814ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 609.546881ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
124432025-03-04T03:29:07.814ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "40626235-1fca-452e-a67b-a98e1f3c079d", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124442025-03-04T03:29:07.816ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 609.219587ms iteration = 23 status = {}
124452025-03-04T03:29:07.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 460 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = d2f71012-f4aa-4ea1-93a8-4a5fdc25382c response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
124462025-03-04T03:29:07.935ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
124472025-03-04T03:29:07.935ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b25bd91c-270e-4515-abfe-ca7bcdecdac1", "content-length": "691", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124482025-03-04T03:29:07.935ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
124492025-03-04T03:29:07.935ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
124502025-03-04T03:29:07.935ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
124512025-03-04T03:29:07.937ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 731.737988ms iteration = 23 status = {}
124522025-03-04T03:29:07.950ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124532025-03-04T03:29:07.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124542025-03-04T03:29:07.952ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
124552025-03-04T03:29:07.952ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124562025-03-04T03:29:07.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124572025-03-04T03:29:07.961ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
124582025-03-04T03:29:07.962ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "38c2cc37-bfca-4692-8e72-48459da524bc", "content-length": "134", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124592025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
124602025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
124612025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
124622025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
124632025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124642025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124652025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124662025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
124672025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124682025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
124692025-03-04T03:29:07.962ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
124702025-03-04T03:29:07.962ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124712025-03-04T03:29:07.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124722025-03-04T03:29:07.964ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
124732025-03-04T03:29:07.965ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "44be8dab-bf35-4796-ac60-995f6d069f27", "content-length": "2", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124742025-03-04T03:29:07.965ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
124752025-03-04T03:29:07.965ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7fd4476-d0ae-449e-b054-602f78a63428", "content-length": "51", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124762025-03-04T03:29:07.965ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
124772025-03-04T03:29:07.965ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
124782025-03-04T03:29:07.985ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
124792025-03-04T03:29:07.985ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
124802025-03-04T03:29:08.020ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
124812025-03-04T03:29:08.020ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:48193
124822025-03-04T03:29:08.020ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:48193 req_id = 812d39ca-1c9b-4196-b06c-d65bb8751d05 roles = RoleSet { roles: {} } uri = /metrics/producers
124832025-03-04T03:29:08.020ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:48193 req_id = 812d39ca-1c9b-4196-b06c-d65bb8751d05 resource = Database result = Ok(()) uri = /metrics/producers
124842025-03-04T03:29:08.060ZINFOnexus: assigned collector to new producer collector_id = ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a file = nexus/src/app/oximeter.rs:76 producer_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
124852025-03-04T03:29:08.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 40395 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:48193 req_id = 812d39ca-1c9b-4196-b06c-d65bb8751d05 response_code = 201 uri = /metrics/producers
124862025-03-04T03:29:08.060ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "812d39ca-1c9b-4196-b06c-d65bb8751d05", "content-length": "41", "date": "Tue, 04 Mar 2025 03:29:08 GMT"} })
124872025-03-04T03:29:08.062ZDEBGnexus (producer-registration-task): registered with nexus successfully
124882025-03-04T03:29:08.062ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
124892025-03-04T03:29:08.062ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ce680414-a359-4d4f-ad56-41e8ba6ece66", "date": "Tue, 04 Mar 2025 03:29:07 GMT"} })
124902025-03-04T03:29:08.062ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
124912025-03-04T03:29:08.062ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
124922025-03-04T03:29:08.063ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "13597726-f1ca-472b-aaba-2a4a9acc4e8d", "content-length": "682", "date": "Tue, 04 Mar 2025 03:29:08 GMT"} })
124932025-03-04T03:29:08.063ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124942025-03-04T03:29:08.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124952025-03-04T03:29:08.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124962025-03-04T03:29:08.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124972025-03-04T03:29:08.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124982025-03-04T03:29:08.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124992025-03-04T03:29:08.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125002025-03-04T03:29:08.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125012025-03-04T03:29:08.080ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 874.884863ms iteration = 23 status = {}
125022025-03-04T03:29:08.783ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8bcf3cc4-f144-48ae-b5f6-e33ac4c131a9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125032025-03-04T03:29:08.783ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8bcf3cc4-f144-48ae-b5f6-e33ac4c131a9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125042025-03-04T03:29:08.787ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4617 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8bcf3cc4-f144-48ae-b5f6-e33ac4c131a9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125052025-03-04T03:29:09.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9658a2f1-1232-4fcd-a3e9-39a2edb697ba roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125062025-03-04T03:29:09.813ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9658a2f1-1232-4fcd-a3e9-39a2edb697ba resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125072025-03-04T03:29:09.815ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2394 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9658a2f1-1232-4fcd-a3e9-39a2edb697ba response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125082025-03-04T03:29:10.834ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b288df42-c22f-4971-aec8-fe79ef7d51e3 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125092025-03-04T03:29:10.835ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b288df42-c22f-4971-aec8-fe79ef7d51e3 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125102025-03-04T03:29:10.837ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2214 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b288df42-c22f-4971-aec8-fe79ef7d51e3 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125112025-03-04T03:29:11.854ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c5222dab-5a85-4af7-946b-31096dfd1f81 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125122025-03-04T03:29:11.855ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c5222dab-5a85-4af7-946b-31096dfd1f81 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125132025-03-04T03:29:11.857ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2141 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c5222dab-5a85-4af7-946b-31096dfd1f81 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125142025-03-04T03:29:12.877ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cb6e227a-577d-4a1d-a04b-af3ff2d26a89 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125152025-03-04T03:29:12.877ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cb6e227a-577d-4a1d-a04b-af3ff2d26a89 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125162025-03-04T03:29:12.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4536 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cb6e227a-577d-4a1d-a04b-af3ff2d26a89 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125172025-03-04T03:29:13.904ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cea6fa2d-077e-448e-b3cf-1e854f48b782 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125182025-03-04T03:29:13.904ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cea6fa2d-077e-448e-b3cf-1e854f48b782 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125192025-03-04T03:29:13.906ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2388 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cea6fa2d-077e-448e-b3cf-1e854f48b782 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125202025-03-04T03:29:14.929ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d21563d-de5a-4d72-859a-fd1250eb6781 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125212025-03-04T03:29:14.929ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d21563d-de5a-4d72-859a-fd1250eb6781 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125222025-03-04T03:29:14.931ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2545 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d21563d-de5a-4d72-859a-fd1250eb6781 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125232025-03-04T03:29:15.951ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22e52f73-d6ec-408d-8d34-a238a7ac9b19 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125242025-03-04T03:29:15.952ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22e52f73-d6ec-408d-8d34-a238a7ac9b19 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125252025-03-04T03:29:15.954ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2258 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 22e52f73-d6ec-408d-8d34-a238a7ac9b19 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125262025-03-04T03:29:16.975ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 023866ff-82b6-4945-ba36-cbbed51f6c6f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125272025-03-04T03:29:16.975ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 023866ff-82b6-4945-ba36-cbbed51f6c6f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125282025-03-04T03:29:16.977ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2611 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 023866ff-82b6-4945-ba36-cbbed51f6c6f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125292025-03-04T03:29:17.205ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
125302025-03-04T03:29:17.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125312025-03-04T03:29:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125322025-03-04T03:29:17.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
125332025-03-04T03:29:17.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125342025-03-04T03:29:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125352025-03-04T03:29:17.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125362025-03-04T03:29:17.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125372025-03-04T03:29:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125382025-03-04T03:29:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
125392025-03-04T03:29:17.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125402025-03-04T03:29:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125412025-03-04T03:29:17.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125422025-03-04T03:29:17.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125432025-03-04T03:29:17.238ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.709099ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
125442025-03-04T03:29:17.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125452025-03-04T03:29:17.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
125462025-03-04T03:29:17.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125472025-03-04T03:29:17.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125482025-03-04T03:29:17.254ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
125492025-03-04T03:29:17.254ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.712497ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
125502025-03-04T03:29:17.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 500 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = aea5c982-65bb-41cc-aed0-3507e07e143a response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
125512025-03-04T03:29:17.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2a0690ee-5212-4320-8a66-c53a5bb5cea2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125522025-03-04T03:29:17.997ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2a0690ee-5212-4320-8a66-c53a5bb5cea2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125532025-03-04T03:29:17.998ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2107 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2a0690ee-5212-4320-8a66-c53a5bb5cea2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125542025-03-04T03:29:19.019ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1496031e-14e8-4dd8-b94e-41b3ac44afb9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125552025-03-04T03:29:19.019ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1496031e-14e8-4dd8-b94e-41b3ac44afb9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125562025-03-04T03:29:19.021ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2157 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1496031e-14e8-4dd8-b94e-41b3ac44afb9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125572025-03-04T03:29:20.013ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 2b9e4940-ad8f-4164-81a6-9fc95270b38e remote_addr = [fd00:1122:3344:101::1]:63537 req_id = adc0e8e6-d38b-4096-8a5d-ba04b9a5b6cc uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-03-04T03:29:19.992329668Z }
125582025-03-04T03:29:20.014ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = adc0e8e6-d38b-4096-8a5d-ba04b9a5b6cc roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
125592025-03-04T03:29:20.014ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = adc0e8e6-d38b-4096-8a5d-ba04b9a5b6cc resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
125602025-03-04T03:29:20.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 77251 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = adc0e8e6-d38b-4096-8a5d-ba04b9a5b6cc response_code = 204 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
125612025-03-04T03:29:20.070ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 2b9e4940-ad8f-4164-81a6-9fc95270b38e remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 044d084f-23bd-4223-bdc1-261732c3065e uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-04T03:29:20.070247803Z }
125622025-03-04T03:29:20.070ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 044d084f-23bd-4223-bdc1-261732c3065e roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
125632025-03-04T03:29:20.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 044d084f-23bd-4223-bdc1-261732c3065e resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
125642025-03-04T03:29:20.108ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 37986 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 044d084f-23bd-4223-bdc1-261732c3065e response_code = 204 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
125652025-03-04T03:29:21.070ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d4192b6-c4ed-489f-9d4e-d566c728b32f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125662025-03-04T03:29:21.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d4192b6-c4ed-489f-9d4e-d566c728b32f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125672025-03-04T03:29:21.073ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2426 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2d4192b6-c4ed-489f-9d4e-d566c728b32f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125682025-03-04T03:29:22.092ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21c37a27-12e0-4860-9152-52b468fd0f6d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125692025-03-04T03:29:22.092ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21c37a27-12e0-4860-9152-52b468fd0f6d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125702025-03-04T03:29:22.094ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2307 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21c37a27-12e0-4860-9152-52b468fd0f6d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125712025-03-04T03:29:23.116ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 440324d4-3edb-466a-870e-136c4b512609 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125722025-03-04T03:29:23.116ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 440324d4-3edb-466a-870e-136c4b512609 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125732025-03-04T03:29:23.118ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2494 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 440324d4-3edb-466a-870e-136c4b512609 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125742025-03-04T03:29:24.140ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7ffcc23-2854-48ac-aed6-4d2d0eece788 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125752025-03-04T03:29:24.140ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7ffcc23-2854-48ac-aed6-4d2d0eece788 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125762025-03-04T03:29:24.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1984 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7ffcc23-2854-48ac-aed6-4d2d0eece788 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125772025-03-04T03:29:25.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 923ab20b-3ddf-4025-a1e8-78ee2cf73bd0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125782025-03-04T03:29:25.161ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 923ab20b-3ddf-4025-a1e8-78ee2cf73bd0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125792025-03-04T03:29:25.163ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2685 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 923ab20b-3ddf-4025-a1e8-78ee2cf73bd0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125802025-03-04T03:29:26.182ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51ed0b7c-6f5f-48f4-9935-69be5072f531 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125812025-03-04T03:29:26.182ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51ed0b7c-6f5f-48f4-9935-69be5072f531 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125822025-03-04T03:29:26.184ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2267 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51ed0b7c-6f5f-48f4-9935-69be5072f531 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125832025-03-04T03:29:27.204ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c391bae8-6ff0-4d4c-88aa-eeecbe4b1269 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125842025-03-04T03:29:27.204ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c391bae8-6ff0-4d4c-88aa-eeecbe4b1269 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125852025-03-04T03:29:27.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
125862025-03-04T03:29:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125872025-03-04T03:29:27.206ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2575 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c391bae8-6ff0-4d4c-88aa-eeecbe4b1269 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
125882025-03-04T03:29:27.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125892025-03-04T03:29:27.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
125902025-03-04T03:29:27.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125912025-03-04T03:29:27.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125922025-03-04T03:29:27.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125932025-03-04T03:29:27.238ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
125942025-03-04T03:29:27.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125952025-03-04T03:29:27.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125962025-03-04T03:29:27.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125972025-03-04T03:29:27.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125982025-03-04T03:29:27.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125992025-03-04T03:29:27.240ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.965304ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
126002025-03-04T03:29:27.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
126012025-03-04T03:29:27.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126022025-03-04T03:29:27.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
126032025-03-04T03:29:27.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
126042025-03-04T03:29:27.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
126052025-03-04T03:29:27.255ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
126062025-03-04T03:29:27.255ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.086802ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
126072025-03-04T03:29:27.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 845 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 34c40e95-2290-4923-bbff-f47067125cc9 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126082025-03-04T03:29:28.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5df32071-b468-409f-94dc-977ca76b38c1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126092025-03-04T03:29:28.227ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5df32071-b468-409f-94dc-977ca76b38c1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126102025-03-04T03:29:28.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2333 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5df32071-b468-409f-94dc-977ca76b38c1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126112025-03-04T03:29:29.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 392ca8e0-b8e3-4d03-97b6-136dde08a7cc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126122025-03-04T03:29:29.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 392ca8e0-b8e3-4d03-97b6-136dde08a7cc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126132025-03-04T03:29:29.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2612 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 392ca8e0-b8e3-4d03-97b6-136dde08a7cc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126142025-03-04T03:29:30.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 144cf3fe-afe7-46b2-b797-f2652b790f23 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126152025-03-04T03:29:30.271ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 144cf3fe-afe7-46b2-b797-f2652b790f23 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126162025-03-04T03:29:30.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1693 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 144cf3fe-afe7-46b2-b797-f2652b790f23 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126172025-03-04T03:29:30.330ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 2b9e4940-ad8f-4164-81a6-9fc95270b38e remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 210d9e64-2029-44ce-93ea-e0b014c15612 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-03-04T03:29:30.330144886Z }
126182025-03-04T03:29:30.330ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 210d9e64-2029-44ce-93ea-e0b014c15612 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126192025-03-04T03:29:30.330ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 210d9e64-2029-44ce-93ea-e0b014c15612 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126202025-03-04T03:29:30.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 39574 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 210d9e64-2029-44ce-93ea-e0b014c15612 response_code = 204 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126212025-03-04T03:29:31.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1158457b-c537-4ec0-a822-e30535b1e5e2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126222025-03-04T03:29:31.293ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1158457b-c537-4ec0-a822-e30535b1e5e2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126232025-03-04T03:29:31.295ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2364 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1158457b-c537-4ec0-a822-e30535b1e5e2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126242025-03-04T03:29:32.314ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4ea77c74-5285-4a5a-936b-a598f265fc29 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126252025-03-04T03:29:32.315ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4ea77c74-5285-4a5a-936b-a598f265fc29 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126262025-03-04T03:29:32.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2387 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4ea77c74-5285-4a5a-936b-a598f265fc29 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
126272025-03-04T03:29:32.767ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 2b9e4940-ad8f-4164-81a6-9fc95270b38e remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-04T03:29:32.767651148Z }
126282025-03-04T03:29:32.768ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126292025-03-04T03:29:32.768ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126302025-03-04T03:29:32.778ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 2b9e4940-ad8f-4164-81a6-9fc95270b38e remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e vmm_needs_update = true
126312025-03-04T03:29:32.779ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126322025-03-04T03:29:32.779ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126332025-03-04T03:29:32.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126342025-03-04T03:29:32.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126352025-03-04T03:29:32.786ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126362025-03-04T03:29:32.786ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126372025-03-04T03:29:32.788ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126382025-03-04T03:29:32.788ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126392025-03-04T03:29:32.817ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126402025-03-04T03:29:32.818ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126412025-03-04T03:29:32.819ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126422025-03-04T03:29:32.819ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126432025-03-04T03:29:32.821ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126442025-03-04T03:29:32.826ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, lookup_type: ById(99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b) }, key: 4aae3d30-75c8-4494-9282-778ebfbc0bf0, lookup_type: ById(4aae3d30-75c8-4494-9282-778ebfbc0bf0) }, key: 6386f39a-ec86-4b2a-bedb-170280f0cece, lookup_type: ById(6386f39a-ec86-4b2a-bedb-170280f0cece) } result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126452025-03-04T03:29:32.834ZINFOnexus (dropshot_internal): starting update saga for 6386f39a-ec86-4b2a-bedb-170280f0cece actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1551 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-04T03:29:32.767651148Z }
126462025-03-04T03:29:32.834ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 66414 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 response_code = 204 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126472025-03-04T03:29:32.834ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 2b9e4940-ad8f-4164-81a6-9fc95270b38e remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f556b4fc-9f8f-4a8c-b78f-a3474b6ac4a4 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-04T03:29:32.767651148Z }
126482025-03-04T03:29:32.834ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f556b4fc-9f8f-4a8c-b78f-a3474b6ac4a4 roles = RoleSet { roles: {} } uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126492025-03-04T03:29:32.835ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f556b4fc-9f8f-4a8c-b78f-a3474b6ac4a4 resource = Database result = Ok(()) uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126502025-03-04T03:29:32.836ZDEBGnexus (dropshot_internal): preparing instance-update saga for 6386f39a-ec86-4b2a-bedb-170280f0cece... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2376a72c-6baf-4436-887f-8af3037bf990 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126512025-03-04T03:29:32.836ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update
126522025-03-04T03:29:32.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4470 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f556b4fc-9f8f-4a8c-b78f-a3474b6ac4a4 response_code = 204 uri = /vmms/2b9e4940-ad8f-4164-81a6-9fc95270b38e
126532025-03-04T03:29:32.841ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"6386f39a-ec86-4b2a-bedb-170280f0cece","lookup_type":{"ById":"6386f39a-ec86-4b2a-bedb-170280f0cece"},"parent":{"key":"4aae3d30-75c8-4494-9282-778ebfbc0bf0","lookup_type":{"ById":"4aae3d30-75c8-4494-9282-778ebfbc0bf0"},"parent":{"key":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","lookup_type":{"ById":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126542025-03-04T03:29:32.841ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update
126552025-03-04T03:29:32.885ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update
126562025-03-04T03:29:32.885ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126572025-03-04T03:29:32.887ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126582025-03-04T03:29:32.887ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
126592025-03-04T03:29:32.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 66578e65-555f-464d-ba91-ee6743bd9576 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
126602025-03-04T03:29:32.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 66578e65-555f-464d-ba91-ee6743bd9576 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
126612025-03-04T03:29:32.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2949 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = 66578e65-555f-464d-ba91-ee6743bd9576 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
126622025-03-04T03:29:32.895ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = ca1cd6a1-d7c8-48bf-ad18-cbdaa3396df4 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
126632025-03-04T03:29:32.895ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = ca1cd6a1-d7c8-48bf-ad18-cbdaa3396df4 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
126642025-03-04T03:29:32.896ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126652025-03-04T03:29:32.896ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
126662025-03-04T03:29:32.897ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2161 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62053 req_id = ca1cd6a1-d7c8-48bf-ad18-cbdaa3396df4 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
126672025-03-04T03:29:32.900ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126682025-03-04T03:29:32.900ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
126692025-03-04T03:29:32.907ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126702025-03-04T03:29:32.907ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("9558995d-9f20-434a-ba5e-08ede7ebee66")) node_id = 0 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
126712025-03-04T03:29:32.912ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126722025-03-04T03:29:32.912ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
126732025-03-04T03:29:32.931ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update
126742025-03-04T03:29:32.933ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126752025-03-04T03:29:32.933ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126762025-03-04T03:29:32.935ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126772025-03-04T03:29:32.935ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126782025-03-04T03:29:32.937ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126792025-03-04T03:29:32.937ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126802025-03-04T03:29:32.940ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126812025-03-04T03:29:32.941ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126822025-03-04T03:29:32.942ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126832025-03-04T03:29:32.942ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126842025-03-04T03:29:32.943ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126852025-03-04T03:29:32.943ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126862025-03-04T03:29:32.945ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126872025-03-04T03:29:32.948ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, lookup_type: ById(99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b) }, key: 4aae3d30-75c8-4494-9282-778ebfbc0bf0, lookup_type: ById(4aae3d30-75c8-4494-9282-778ebfbc0bf0) }, key: 6386f39a-ec86-4b2a-bedb-170280f0cece, lookup_type: ById(6386f39a-ec86-4b2a-bedb-170280f0cece) } result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126882025-03-04T03:29:32.951ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance updater_id = 9558995d-9f20-434a-ba5e-08ede7ebee66
126892025-03-04T03:29:32.951ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126902025-03-04T03:29:32.951ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126912025-03-04T03:29:32.968ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance updater_id = 9558995d-9f20-434a-ba5e-08ede7ebee66
126922025-03-04T03:29:32.968ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126932025-03-04T03:29:32.968ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance
126942025-03-04T03:29:32.975ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece locked_gen = Generation(Generation(2)) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = LockInstance updater_id = 9558995d-9f20-434a-ba5e-08ede7ebee66
126952025-03-04T03:29:32.975ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126962025-03-04T03:29:32.975ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("9558995d-9f20-434a-ba5e-08ede7ebee66")}) node_id = 1 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
126972025-03-04T03:29:32.979ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
126982025-03-04T03:29:32.979ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
126992025-03-04T03:29:32.987ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127002025-03-04T03:29:32.988ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127012025-03-04T03:29:32.989ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127022025-03-04T03:29:32.989ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127032025-03-04T03:29:32.991ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127042025-03-04T03:29:32.991ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127052025-03-04T03:29:32.992ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127062025-03-04T03:29:32.995ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, lookup_type: ById(99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b) }, key: 4aae3d30-75c8-4494-9282-778ebfbc0bf0, lookup_type: ById(4aae3d30-75c8-4494-9282-778ebfbc0bf0) }, key: 6386f39a-ec86-4b2a-bedb-170280f0cece, lookup_type: ById(6386f39a-ec86-4b2a-bedb-170280f0cece) } result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127072025-03-04T03:29:32.995ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127082025-03-04T03:29:32.995ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
127092025-03-04T03:29:33.005ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 2b9e4940-ad8f-4164-81a6-9fc95270b38e, time_created: 2025-03-04T03:29:11.994457Z, time_deleted: None, instance_id: 6386f39a-ec86-4b2a-bedb-170280f0cece, sled_id: 5525e426-7299-43ab-ad2b-346968b24e1c, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-03-04T03:29:32.767651Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-03-04T03:29:09.900139Z, gen: Generation(Generation(3)), propolis_id: Some(2b9e4940-ad8f-4164-81a6-9fc95270b38e), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(2b9e4940-ad8f-4164-81a6-9fc95270b38e (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-03-04T03:29:33.005282771Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
127102025-03-04T03:29:33.009ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127112025-03-04T03:29:33.009ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"6386f39a-ec86-4b2a-bedb-170280f0cece","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"2b9e4940-ad8f-4164-81a6-9fc95270b38e"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"6386f39a-ec86-4b2a-bedb-170280f0cece","lookup_type":{"ById":"6386f39a-ec86-4b2a-bedb-170280f0cece"},"parent":{"key":"4aae3d30-75c8-4494-9282-778ebfbc0bf0","lookup_type":{"ById":"4aae3d30-75c8-4494-9282-778ebfbc0bf0"},"parent":{"key":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","lookup_type":{"ById":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"9558995d-9f20-434a-ba5e-08ede7ebee66"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"4aae3d30-75c8-4494-9282-778ebfbc0bf0","ram_diff":1073741824},"destroy_active_vmm":"2b9e4940-ad8f-4164-81a6-9fc95270b38e","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-03-04T03:29:33.005282771Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127122025-03-04T03:29:33.009ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127132025-03-04T03:29:33.041ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127142025-03-04T03:29:33.041ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127152025-03-04T03:29:33.041ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127162025-03-04T03:29:33.041ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127172025-03-04T03:29:33.045ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127182025-03-04T03:29:33.045ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127192025-03-04T03:29:33.048ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127202025-03-04T03:29:33.048ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127212025-03-04T03:29:33.052ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127222025-03-04T03:29:33.052ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("13ce3aa7-2be8-496f-a608-93b6f1658ad0")) node_id = 0 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127232025-03-04T03:29:33.056ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127242025-03-04T03:29:33.056ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127252025-03-04T03:29:33.062ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece parent_lock = UpdaterLock { updater_id: 9558995d-9f20-434a-ba5e-08ede7ebee66, locked_gen: Generation(Generation(2)) } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127262025-03-04T03:29:33.062ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = BecomeUpdater
127272025-03-04T03:29:33.062ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = BecomeUpdater
127282025-03-04T03:29:33.073ZDEBGnexus (SagaExecutor): inherited lock from 9558995d-9f20-434a-ba5e-08ede7ebee66 to 13ce3aa7-2be8-496f-a608-93b6f1658ad0 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 9558995d-9f20-434a-ba5e-08ede7ebee66 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = BecomeUpdater updater_id = 13ce3aa7-2be8-496f-a608-93b6f1658ad0
127292025-03-04T03:29:33.073ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127302025-03-04T03:29:33.073ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127312025-03-04T03:29:33.073ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("13ce3aa7-2be8-496f-a608-93b6f1658ad0")}) node_id = 1 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127322025-03-04T03:29:33.076ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127332025-03-04T03:29:33.076ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127342025-03-04T03:29:33.079ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127352025-03-04T03:29:33.079ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127362025-03-04T03:29:33.082ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127372025-03-04T03:29:33.082ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127382025-03-04T03:29:33.089ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127392025-03-04T03:29:33.089ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece
127402025-03-04T03:29:33.092ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UpdateNetworkConfig
127412025-03-04T03:29:33.092ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UpdateNetworkConfig
127422025-03-04T03:29:33.099ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UpdateNetworkConfig
127432025-03-04T03:29:33.099ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UpdateNetworkConfig
127442025-03-04T03:29:33.101ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
127452025-03-04T03:29:33.101ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UpdateNetworkConfig
127462025-03-04T03:29:33.101ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UpdateNetworkConfig
127472025-03-04T03:29:33.104ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
127482025-03-04T03:29:33.104ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
127492025-03-04T03:29:33.104ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
127502025-03-04T03:29:33.105ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
127512025-03-04T03:29:33.106ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
127522025-03-04T03:29:33.106ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
127532025-03-04T03:29:33.106ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
127542025-03-04T03:29:33.110ZDEBGnexus: notifying dendrite of updates instance_id = Some(6386f39a-ec86-4b2a-bedb-170280f0cece (instance)) switch = switch0
127552025-03-04T03:29:33.111ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1685, tv_nsec: 305781634 } })
127562025-03-04T03:29:33.111ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1155
127572025-03-04T03:29:33.141ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
127582025-03-04T03:29:33.141ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
127592025-03-04T03:29:33.141ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c897e742-e945-4e5b-9838-df93dcb4034d", "content-length": "26", "date": "Tue, 04 Mar 2025 03:29:33 GMT"} })
127602025-03-04T03:29:33.141ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
127612025-03-04T03:29:33.142ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
127622025-03-04T03:29:33.171ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
127632025-03-04T03:29:33.171ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef88baef-e607-4527-bdb9-4958de571b27", "content-length": "4", "date": "Tue, 04 Mar 2025 03:29:32 GMT"} })
127642025-03-04T03:29:33.173ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127652025-03-04T03:29:33.173ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127662025-03-04T03:29:33.176ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127672025-03-04T03:29:33.176ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127682025-03-04T03:29:33.182ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ReleaseVirtualProvisioning
127692025-03-04T03:29:33.182ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ReleaseVirtualProvisioning
127702025-03-04T03:29:33.257ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-03-04T03:29:33.185195Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 4aae3d30-75c8-4494-9282-778ebfbc0bf0, time_modified: Some(2025-03-04T03:29:33.185195Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, time_modified: Some(2025-03-04T03:29:33.185195Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127712025-03-04T03:29:33.257ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127722025-03-04T03:29:33.257ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127732025-03-04T03:29:33.261ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127742025-03-04T03:29:33.261ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127752025-03-04T03:29:33.266ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127762025-03-04T03:29:33.266ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UnassignOximeterProducer
127772025-03-04T03:29:33.266ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UnassignOximeterProducer
127782025-03-04T03:29:33.275ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a producer_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127792025-03-04T03:29:33.275ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UnassignOximeterProducer
127802025-03-04T03:29:33.276ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = UnassignOximeterProducer
127812025-03-04T03:29:33.317ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127822025-03-04T03:29:33.317ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/6386f39a-ec86-4b2a-bedb-170280f0cece
127832025-03-04T03:29:33.338ZDEBGnexus (SagaExecutor): client response oximeter-collector = 6386f39a-ec86-4b2a-bedb-170280f0cece result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/6386f39a-ec86-4b2a-bedb-170280f0cece", status: 204, headers: {"x-request-id": "c6d92939-30fd-46bc-8b17-7eedf4fbf17e", "date": "Tue, 04 Mar 2025 03:29:33 GMT"} }) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127842025-03-04T03:29:33.338ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a producer_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127852025-03-04T03:29:33.340ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127862025-03-04T03:29:33.340ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127872025-03-04T03:29:33.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5d699c08-9f9a-4463-b787-2ea08e26794e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
127882025-03-04T03:29:33.340ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5d699c08-9f9a-4463-b787-2ea08e26794e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
127892025-03-04T03:29:33.342ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2756 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5d699c08-9f9a-4463-b787-2ea08e26794e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
127902025-03-04T03:29:33.351ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127912025-03-04T03:29:33.351ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127922025-03-04T03:29:33.357ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece lock = UpdaterLock { updater_id: 13ce3aa7-2be8-496f-a608-93b6f1658ad0, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2025-03-04T03:29:33.005282771Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127932025-03-04T03:29:33.357ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = CommitInstanceUpdates
127942025-03-04T03:29:33.357ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = CommitInstanceUpdates
127952025-03-04T03:29:33.401ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece new_runtime = InstanceRuntimeState { time_updated: 2025-03-04T03:29:33.005282771Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
127962025-03-04T03:29:33.401ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
127972025-03-04T03:29:33.401ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
127982025-03-04T03:29:33.401ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Signaled
127992025-03-04T03:29:33.401ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Signaled
128002025-03-04T03:29:33.401ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
128012025-03-04T03:29:33.401ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
128022025-03-04T03:29:33.401ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128032025-03-04T03:29:33.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128042025-03-04T03:29:33.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
128052025-03-04T03:29:33.403ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128062025-03-04T03:29:33.405ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
128072025-03-04T03:29:33.405ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128082025-03-04T03:29:33.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128092025-03-04T03:29:33.405ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128102025-03-04T03:29:33.405ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128112025-03-04T03:29:33.406ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128122025-03-04T03:29:33.408ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
128132025-03-04T03:29:33.408ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128142025-03-04T03:29:33.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128152025-03-04T03:29:33.410ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128162025-03-04T03:29:33.410ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("6386f39a-ec86-4b2a-bedb-170280f0cece"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("2b9e4940-ad8f-4164-81a6-9fc95270b38e")}) node_id = 7 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128172025-03-04T03:29:33.414ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128182025-03-04T03:29:33.415ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128192025-03-04T03:29:33.418ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128202025-03-04T03:29:33.418ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128212025-03-04T03:29:33.422ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128222025-03-04T03:29:33.422ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128232025-03-04T03:29:33.425ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:82 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece propolis_id = 2b9e4940-ad8f-4164-81a6-9fc95270b38e saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
128242025-03-04T03:29:33.425ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ReleaseSledResources
128252025-03-04T03:29:33.425ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ReleaseSledResources
128262025-03-04T03:29:33.440ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
128272025-03-04T03:29:33.440ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
128282025-03-04T03:29:33.441ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3db01e80-f384-4b6e-a0df-5244500a54d1", "content-length": "372", "date": "Tue, 04 Mar 2025 03:29:32 GMT"} })
128292025-03-04T03:29:33.441ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128302025-03-04T03:29:33.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128312025-03-04T03:29:33.453ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128322025-03-04T03:29:33.453ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128332025-03-04T03:29:33.456ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128342025-03-04T03:29:33.456ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128352025-03-04T03:29:33.460ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:113 instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece propolis_id = 2b9e4940-ad8f-4164-81a6-9fc95270b38e saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
128362025-03-04T03:29:33.460ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = MarkVmmDeleted
128372025-03-04T03:29:33.460ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = MarkVmmDeleted
128382025-03-04T03:29:33.472ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128392025-03-04T03:29:33.472ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128402025-03-04T03:29:33.475ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128412025-03-04T03:29:33.475ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128422025-03-04T03:29:33.479ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128432025-03-04T03:29:33.479ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128442025-03-04T03:29:33.482ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128452025-03-04T03:29:33.482ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128462025-03-04T03:29:33.484ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {77736455-ed72-486f-ba51-f777183a49b1: {10.151.2.176: {38c86375-c330-4b97-a512-a88012ce3a75}, 10.151.2.175: {38c86375-c330-4b97-a512-a88012ce3a75}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
128472025-03-04T03:29:33.484ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
128482025-03-04T03:29:33.484ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f9acca7c-30b3-4dfa-9215-78352c4edb5f", "date": "Tue, 04 Mar 2025 03:29:32 GMT"} })
128492025-03-04T03:29:33.484ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128502025-03-04T03:29:33.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128512025-03-04T03:29:33.486ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128522025-03-04T03:29:33.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128532025-03-04T03:29:33.488ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128542025-03-04T03:29:33.488ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128552025-03-04T03:29:33.490ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128562025-03-04T03:29:33.490ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128572025-03-04T03:29:33.490ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128582025-03-04T03:29:33.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128592025-03-04T03:29:33.493ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
128602025-03-04T03:29:33.493ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
128612025-03-04T03:29:33.493ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "dc566f62-99e5-4b71-bad5-e32fa042f439", "date": "Tue, 04 Mar 2025 03:29:32 GMT"} })
128622025-03-04T03:29:33.495ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 94.043582ms iteration = 24 status = {}
128632025-03-04T03:29:33.496ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
128642025-03-04T03:29:33.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
128652025-03-04T03:29:33.519ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128662025-03-04T03:29:33.519ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128672025-03-04T03:29:33.521ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128682025-03-04T03:29:33.524ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, lookup_type: ById(99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b) }, key: 4aae3d30-75c8-4494-9282-778ebfbc0bf0, lookup_type: ById(4aae3d30-75c8-4494-9282-778ebfbc0bf0) }, key: 6386f39a-ec86-4b2a-bedb-170280f0cece, lookup_type: ById(6386f39a-ec86-4b2a-bedb-170280f0cece) } result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128692025-03-04T03:29:33.524ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128702025-03-04T03:29:33.524ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_node = ChainSuccessorSaga
128712025-03-04T03:29:33.529ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece karmic_state = WillReincarnate needs_reincarnation = false saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update
128722025-03-04T03:29:33.529ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128732025-03-04T03:29:33.529ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128742025-03-04T03:29:33.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
128752025-03-04T03:29:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
128762025-03-04T03:29:33.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128772025-03-04T03:29:33.532ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128782025-03-04T03:29:33.532ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128792025-03-04T03:29:33.533ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
128802025-03-04T03:29:33.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
128812025-03-04T03:29:33.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
128822025-03-04T03:29:33.534ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128832025-03-04T03:29:33.536ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
128842025-03-04T03:29:33.536ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
128852025-03-04T03:29:33.536ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128862025-03-04T03:29:33.536ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128872025-03-04T03:29:33.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
128882025-03-04T03:29:33.539ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128892025-03-04T03:29:33.539ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a
128902025-03-04T03:29:33.549ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128912025-03-04T03:29:33.549ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a saga_name = instance-update saga_result = SagaResult { saga_id: 871e21b8-13fd-474b-9882-cf39941cee7a, saga_log: SagaLog { saga_id: 871e21b8-13fd-474b-9882-cf39941cee7a, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("13ce3aa7-2be8-496f-a608-93b6f1658ad0")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("13ce3aa7-2be8-496f-a608-93b6f1658ad0")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("6386f39a-ec86-4b2a-bedb-170280f0cece"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("2b9e4940-ad8f-4164-81a6-9fc95270b38e")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("6386f39a-ec86-4b2a-bedb-170280f0cece"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("2b9e4940-ad8f-4164-81a6-9fc95270b38e")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("13ce3aa7-2be8-496f-a608-93b6f1658ad0"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("13ce3aa7-2be8-496f-a608-93b6f1658ad0")}} }) }
128922025-03-04T03:29:33.549ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 871e21b8-13fd-474b-9882-cf39941cee7a instance_id = 6386f39a-ec86-4b2a-bedb-170280f0cece saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update
128932025-03-04T03:29:33.549ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128942025-03-04T03:29:33.549ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
128952025-03-04T03:29:33.552ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128962025-03-04T03:29:33.552ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
128972025-03-04T03:29:33.556ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
128982025-03-04T03:29:33.556ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
128992025-03-04T03:29:33.559ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
129002025-03-04T03:29:33.559ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2
129012025-03-04T03:29:33.569ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
129022025-03-04T03:29:33.569ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "157f031d-00b8-40c0-9b1e-a682ee5c8cd9", "content-length": "691", "date": "Tue, 04 Mar 2025 03:29:32 GMT"} })
129032025-03-04T03:29:33.569ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
129042025-03-04T03:29:33.569ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
129052025-03-04T03:29:33.569ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
129062025-03-04T03:29:33.570ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 169.468537ms iteration = 24 status = {}
129072025-03-04T03:29:33.610ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update sec_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
129082025-03-04T03:29:33.610ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2 saga_name = start-instance-update saga_result = SagaResult { saga_id: 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2, saga_log: SagaLog { saga_id: 8ea03af3-1cb9-4abe-a452-f2cd5054d2d2, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("9558995d-9f20-434a-ba5e-08ede7ebee66")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("9558995d-9f20-434a-ba5e-08ede7ebee66")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("9558995d-9f20-434a-ba5e-08ede7ebee66"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("9558995d-9f20-434a-ba5e-08ede7ebee66")}} }) }
129092025-03-04T03:29:34.365ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9ab1ce03-4ab4-4cf6-9acf-e9df5f8a4fff roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129102025-03-04T03:29:34.365ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9ab1ce03-4ab4-4cf6-9acf-e9df5f8a4fff resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129112025-03-04T03:29:34.367ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1799 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9ab1ce03-4ab4-4cf6-9acf-e9df5f8a4fff response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129122025-03-04T03:29:35.387ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12887bc0-a4a8-4ac6-8279-d47478c00882 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129132025-03-04T03:29:35.387ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12887bc0-a4a8-4ac6-8279-d47478c00882 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129142025-03-04T03:29:35.389ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1834 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 12887bc0-a4a8-4ac6-8279-d47478c00882 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129152025-03-04T03:29:36.409ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b25ef985-69d9-4d05-b76c-535517baf80a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129162025-03-04T03:29:36.409ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b25ef985-69d9-4d05-b76c-535517baf80a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129172025-03-04T03:29:36.411ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2325 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b25ef985-69d9-4d05-b76c-535517baf80a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
129182025-03-04T03:29:37.205ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
129192025-03-04T03:29:37.205ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
129202025-03-04T03:29:37.205ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
129212025-03-04T03:29:37.205ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
129222025-03-04T03:29:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
129232025-03-04T03:29:37.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
129242025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
129252025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
129262025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
129272025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
129282025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
129292025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
129302025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
129312025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
129322025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
129332025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
129342025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
129352025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
129362025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
129372025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
129382025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129392025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
129402025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
129412025-03-04T03:29:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
129422025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
129432025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
129442025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
129452025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
129462025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
129472025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
129482025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
129492025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
129502025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
129512025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
129522025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
129532025-03-04T03:29:37.207ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
129542025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
129552025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
129562025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
129572025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
129582025-03-04T03:29:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
129592025-03-04T03:29:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
129602025-03-04T03:29:37.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1689, tv_nsec: 403465605 } })
129612025-03-04T03:29:37.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1689, tv_nsec: 403539285 } })
129622025-03-04T03:29:37.209ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
129632025-03-04T03:29:37.209ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
129642025-03-04T03:29:37.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129652025-03-04T03:29:37.210ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.708185ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
129662025-03-04T03:29:37.210ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129672025-03-04T03:29:37.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129682025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
129692025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
129702025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
129712025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
129722025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
129732025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
129742025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
129752025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
129762025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
129772025-03-04T03:29:37.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
129782025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
129792025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
129802025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
129812025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
129822025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
129832025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
129842025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
129852025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
129862025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
129872025-03-04T03:29:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
129882025-03-04T03:29:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
129892025-03-04T03:29:37.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
129902025-03-04T03:29:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
129912025-03-04T03:29:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
129922025-03-04T03:29:37.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
129932025-03-04T03:29:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
129942025-03-04T03:29:37.214ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129952025-03-04T03:29:37.215ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129962025-03-04T03:29:37.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129972025-03-04T03:29:37.216ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
129982025-03-04T03:29:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
129992025-03-04T03:29:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
130002025-03-04T03:29:37.216ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
130012025-03-04T03:29:37.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
130022025-03-04T03:29:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
130032025-03-04T03:29:37.217ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
130042025-03-04T03:29:37.217ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
130052025-03-04T03:29:37.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
130062025-03-04T03:29:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
130072025-03-04T03:29:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
130082025-03-04T03:29:37.221ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
130092025-03-04T03:29:37.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
130102025-03-04T03:29:37.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130112025-03-04T03:29:37.222ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.609999ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
130122025-03-04T03:29:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130132025-03-04T03:29:37.222ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1689, tv_nsec: 417273154 } })
130142025-03-04T03:29:37.223ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
130152025-03-04T03:29:37.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130162025-03-04T03:29:37.228ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
130172025-03-04T03:29:37.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130182025-03-04T03:29:37.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130192025-03-04T03:29:37.254ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130202025-03-04T03:29:37.256ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
130212025-03-04T03:29:37.256ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
130222025-03-04T03:29:37.256ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
130232025-03-04T03:29:37.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
130242025-03-04T03:29:37.256ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
130252025-03-04T03:29:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
130262025-03-04T03:29:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
130272025-03-04T03:29:37.257ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
130282025-03-04T03:29:37.257ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 51.114077ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
130292025-03-04T03:29:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
130302025-03-04T03:29:37.258ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
130312025-03-04T03:29:37.258ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 51.597822ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
130322025-03-04T03:29:37.259ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
130332025-03-04T03:29:37.260ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
130342025-03-04T03:29:37.260ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
130352025-03-04T03:29:37.260ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
130362025-03-04T03:29:37.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
130372025-03-04T03:29:37.260ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
130382025-03-04T03:29:37.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
130392025-03-04T03:29:37.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
130402025-03-04T03:29:37.260ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
130412025-03-04T03:29:37.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
130422025-03-04T03:29:37.262ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.067986ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
130432025-03-04T03:29:37.263ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130442025-03-04T03:29:37.265ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
130452025-03-04T03:29:37.265ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 57.547926ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
130462025-03-04T03:29:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
130472025-03-04T03:29:37.265ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
130482025-03-04T03:29:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
130492025-03-04T03:29:37.266ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 54.215875ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
130502025-03-04T03:29:37.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 54.68599ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
130512025-03-04T03:29:37.266ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
130522025-03-04T03:29:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
130532025-03-04T03:29:37.270ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130542025-03-04T03:29:37.272ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
130552025-03-04T03:29:37.272ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
130562025-03-04T03:29:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
130572025-03-04T03:29:37.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
130582025-03-04T03:29:37.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
130592025-03-04T03:29:37.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130602025-03-04T03:29:37.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
130612025-03-04T03:29:37.284ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
130622025-03-04T03:29:37.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
130632025-03-04T03:29:37.296ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130642025-03-04T03:29:37.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 85.555129ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
130652025-03-04T03:29:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
130662025-03-04T03:29:37.298ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
130672025-03-04T03:29:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
130682025-03-04T03:29:37.299ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
130692025-03-04T03:29:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
130702025-03-04T03:29:37.300ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 94.261575ms iteration = 24 status = 0
130712025-03-04T03:29:37.302ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
130722025-03-04T03:29:37.302ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
130732025-03-04T03:29:37.302ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 95.008745ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
130742025-03-04T03:29:37.308ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
130752025-03-04T03:29:37.308ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
130762025-03-04T03:29:37.308ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4924090d-902d-49b0-ab41-8881787a4ca1", "content-length": "26", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
130772025-03-04T03:29:37.308ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
130782025-03-04T03:29:37.310ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
130792025-03-04T03:29:37.316ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
130802025-03-04T03:29:37.316ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 109.684324ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
130812025-03-04T03:29:37.331ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
130822025-03-04T03:29:37.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
130832025-03-04T03:29:37.353ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
130842025-03-04T03:29:37.353ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
130852025-03-04T03:29:37.353ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad6b5521-6578-413f-b785-95c7bbbb5abf", "content-length": "26", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
130862025-03-04T03:29:37.353ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
130872025-03-04T03:29:37.355ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
130882025-03-04T03:29:37.356ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 150.14064ms iteration = 27 status = {"physical_disks_added":0}
130892025-03-04T03:29:37.362ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
130902025-03-04T03:29:37.362ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
130912025-03-04T03:29:37.362ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "147678cd-ae6d-46f5-bbde-ae9bc10e36ea", "content-length": "26", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
130922025-03-04T03:29:37.362ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
130932025-03-04T03:29:37.364ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
130942025-03-04T03:29:37.430ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 89be1caf-a9b5-4d1a-b88b-f63144969a3c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
130952025-03-04T03:29:37.430ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 89be1caf-a9b5-4d1a-b88b-f63144969a3c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
130962025-03-04T03:29:37.431ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
130972025-03-04T03:29:37.431ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
130982025-03-04T03:29:37.431ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "14ee5efa-1c21-4df0-b3dd-798e6754313a", "content-length": "372", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
130992025-03-04T03:29:37.431ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131002025-03-04T03:29:37.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
131012025-03-04T03:29:37.433ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 221.364893ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
131022025-03-04T03:29:37.434ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4085 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 89be1caf-a9b5-4d1a-b88b-f63144969a3c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131032025-03-04T03:29:37.438ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
131042025-03-04T03:29:37.439ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a714a10e-9644-4497-a957-e5b0f234223b", "content-length": "2", "date": "Tue, 04 Mar 2025 03:29:36 GMT"} })
131052025-03-04T03:29:37.439ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
131062025-03-04T03:29:37.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
131072025-03-04T03:29:37.441ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131082025-03-04T03:29:37.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
131092025-03-04T03:29:37.441ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 236.228175ms iteration = 24 status = null
131102025-03-04T03:29:37.443ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131112025-03-04T03:29:37.445ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
131122025-03-04T03:29:37.445ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131132025-03-04T03:29:37.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
131142025-03-04T03:29:37.447ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131152025-03-04T03:29:37.448ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
131162025-03-04T03:29:37.448ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
131172025-03-04T03:29:37.448ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
131182025-03-04T03:29:37.448ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131192025-03-04T03:29:37.449ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3e06e0bb-03ac-4393-b15e-d1f496a84154", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
131202025-03-04T03:29:37.449ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131212025-03-04T03:29:37.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
131222025-03-04T03:29:37.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
131232025-03-04T03:29:37.451ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131242025-03-04T03:29:37.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
131252025-03-04T03:29:37.454ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131262025-03-04T03:29:37.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
131272025-03-04T03:29:37.457ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
131282025-03-04T03:29:37.457ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
131292025-03-04T03:29:37.457ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "004ab9a4-c690-4b15-8685-437053153fb8", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
131302025-03-04T03:29:37.459ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 251.607443ms iteration = 25 status = {}
131312025-03-04T03:29:37.537ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
131322025-03-04T03:29:37.538ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a79e621b-5c84-4e16-9f7e-457e044b4b92", "content-length": "691", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
131332025-03-04T03:29:37.538ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
131342025-03-04T03:29:37.538ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
131352025-03-04T03:29:37.538ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
131362025-03-04T03:29:37.538ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 332.426744ms iteration = 25 status = {}
131372025-03-04T03:29:37.546ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 341.240073ms iteration = 24 status = {}
131382025-03-04T03:29:37.604ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131392025-03-04T03:29:37.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131402025-03-04T03:29:37.606ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
131412025-03-04T03:29:37.606ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131422025-03-04T03:29:37.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131432025-03-04T03:29:37.616ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
131442025-03-04T03:29:37.616ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c1b66aa-4736-401b-9b7a-7dd5da575f44", "content-length": "134", "date": "Tue, 04 Mar 2025 03:29:36 GMT"} })
131452025-03-04T03:29:37.616ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
131462025-03-04T03:29:37.616ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
131472025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
131482025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
131492025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
131502025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
131512025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
131522025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
131532025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
131542025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
131552025-03-04T03:29:37.617ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
131562025-03-04T03:29:37.617ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131572025-03-04T03:29:37.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131582025-03-04T03:29:37.619ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
131592025-03-04T03:29:37.619ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5035b6d-ef77-4562-aed0-2d3a54c2ad69", "content-length": "2", "date": "Tue, 04 Mar 2025 03:29:36 GMT"} })
131602025-03-04T03:29:37.619ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
131612025-03-04T03:29:37.619ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a448859-41c6-47c3-bb33-8c9b7b6f48b9", "content-length": "51", "date": "Tue, 04 Mar 2025 03:29:36 GMT"} })
131622025-03-04T03:29:37.619ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
131632025-03-04T03:29:37.619ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
131642025-03-04T03:29:37.709ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d4a75054-1d11-4902-826a-2afb925918d6", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
131652025-03-04T03:29:37.709ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
131662025-03-04T03:29:37.709ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
131672025-03-04T03:29:37.709ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "842981c3-8007-40b0-a073-e7ec194e3abf", "content-length": "682", "date": "Tue, 04 Mar 2025 03:29:37 GMT"} })
131682025-03-04T03:29:37.709ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131692025-03-04T03:29:37.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131702025-03-04T03:29:37.713ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131712025-03-04T03:29:37.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131722025-03-04T03:29:37.717ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131732025-03-04T03:29:37.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131742025-03-04T03:29:37.720ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131752025-03-04T03:29:37.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131762025-03-04T03:29:37.728ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 521.701824ms iteration = 24 status = {}
131772025-03-04T03:29:37.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 533 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = bd7713f3-994f-48ac-886d-3c5b16628a20 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
131782025-03-04T03:29:38.453ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8123f48a-3348-4611-b295-6d75aa1f373b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131792025-03-04T03:29:38.454ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8123f48a-3348-4611-b295-6d75aa1f373b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131802025-03-04T03:29:38.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2440 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8123f48a-3348-4611-b295-6d75aa1f373b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131812025-03-04T03:29:39.475ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0cd0ba85-00fe-40cd-b4b0-7c23dff10ef6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131822025-03-04T03:29:39.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0cd0ba85-00fe-40cd-b4b0-7c23dff10ef6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131832025-03-04T03:29:39.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2232 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0cd0ba85-00fe-40cd-b4b0-7c23dff10ef6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131842025-03-04T03:29:40.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd6a4bde-1838-4ce8-97ea-7ec5638bfffd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131852025-03-04T03:29:40.497ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd6a4bde-1838-4ce8-97ea-7ec5638bfffd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131862025-03-04T03:29:40.499ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2490 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fd6a4bde-1838-4ce8-97ea-7ec5638bfffd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131872025-03-04T03:29:41.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1946da4f-8a83-405d-80c4-3619b0cca2e9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131882025-03-04T03:29:41.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1946da4f-8a83-405d-80c4-3619b0cca2e9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131892025-03-04T03:29:41.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2486 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1946da4f-8a83-405d-80c4-3619b0cca2e9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131902025-03-04T03:29:42.540ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5ec47d91-087a-4916-95c5-0aaa379c81cc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131912025-03-04T03:29:42.540ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5ec47d91-087a-4916-95c5-0aaa379c81cc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131922025-03-04T03:29:42.542ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2366 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5ec47d91-087a-4916-95c5-0aaa379c81cc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131932025-03-04T03:29:43.562ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2450b9ad-71cc-48a8-8e47-eab18f10a194 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131942025-03-04T03:29:43.562ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2450b9ad-71cc-48a8-8e47-eab18f10a194 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131952025-03-04T03:29:43.565ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2920 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2450b9ad-71cc-48a8-8e47-eab18f10a194 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131962025-03-04T03:29:44.584ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0628447e-66ee-4019-9125-f1735fb9fefa roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131972025-03-04T03:29:44.584ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0628447e-66ee-4019-9125-f1735fb9fefa resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131982025-03-04T03:29:44.589ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 5368 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0628447e-66ee-4019-9125-f1735fb9fefa response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
131992025-03-04T03:29:45.608ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04431702-6b02-4ace-b1d6-37799e739d36 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132002025-03-04T03:29:45.609ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04431702-6b02-4ace-b1d6-37799e739d36 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132012025-03-04T03:29:45.611ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2501 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04431702-6b02-4ace-b1d6-37799e739d36 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132022025-03-04T03:29:46.630ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5ad167f3-08d1-44ab-b4ef-74d0d82b4e87 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132032025-03-04T03:29:46.630ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5ad167f3-08d1-44ab-b4ef-74d0d82b4e87 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132042025-03-04T03:29:46.632ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1646 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 5ad167f3-08d1-44ab-b4ef-74d0d82b4e87 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132052025-03-04T03:29:47.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
132062025-03-04T03:29:47.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132072025-03-04T03:29:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132082025-03-04T03:29:47.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
132092025-03-04T03:29:47.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
132102025-03-04T03:29:47.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132112025-03-04T03:29:47.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132122025-03-04T03:29:47.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
132132025-03-04T03:29:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132142025-03-04T03:29:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
132152025-03-04T03:29:47.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132162025-03-04T03:29:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132172025-03-04T03:29:47.211ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.07385ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
132182025-03-04T03:29:47.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132192025-03-04T03:29:47.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132202025-03-04T03:29:47.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132212025-03-04T03:29:47.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
132222025-03-04T03:29:47.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132232025-03-04T03:29:47.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132242025-03-04T03:29:47.229ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
132252025-03-04T03:29:47.229ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.202335ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
132262025-03-04T03:29:47.651ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21f1917a-af54-4b76-ad4d-b83d48237545 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132272025-03-04T03:29:47.651ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21f1917a-af54-4b76-ad4d-b83d48237545 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132282025-03-04T03:29:47.653ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2150 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21f1917a-af54-4b76-ad4d-b83d48237545 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132292025-03-04T03:29:47.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 578 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 740fea90-cc13-4a98-956d-dd99b1d1be86 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
132302025-03-04T03:29:48.672ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 39e981aa-22bc-4b5c-abad-1be73bd7be81 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132312025-03-04T03:29:48.673ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 39e981aa-22bc-4b5c-abad-1be73bd7be81 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132322025-03-04T03:29:48.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2167 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 39e981aa-22bc-4b5c-abad-1be73bd7be81 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132332025-03-04T03:29:49.696ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9770d17c-8459-4b42-b091-4b1451309fba roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132342025-03-04T03:29:49.696ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9770d17c-8459-4b42-b091-4b1451309fba resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132352025-03-04T03:29:49.698ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2643 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9770d17c-8459-4b42-b091-4b1451309fba response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132362025-03-04T03:29:50.719ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ffb9d1b8-0a35-4b9b-af5d-01ea08f5b24c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132372025-03-04T03:29:50.719ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ffb9d1b8-0a35-4b9b-af5d-01ea08f5b24c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132382025-03-04T03:29:50.721ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2237 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ffb9d1b8-0a35-4b9b-af5d-01ea08f5b24c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132392025-03-04T03:29:51.741ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f4d775c-f12d-4dc4-8795-8c07be6c52eb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132402025-03-04T03:29:51.741ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f4d775c-f12d-4dc4-8795-8c07be6c52eb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132412025-03-04T03:29:51.743ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2429 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7f4d775c-f12d-4dc4-8795-8c07be6c52eb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132422025-03-04T03:29:52.763ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bfc7b520-b442-4afe-ae18-a33abb3cf62f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132432025-03-04T03:29:52.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bfc7b520-b442-4afe-ae18-a33abb3cf62f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132442025-03-04T03:29:52.765ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2478 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bfc7b520-b442-4afe-ae18-a33abb3cf62f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132452025-03-04T03:29:53.786ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42d72d6c-462b-4482-b42a-3b1293e8dd92 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132462025-03-04T03:29:53.787ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42d72d6c-462b-4482-b42a-3b1293e8dd92 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132472025-03-04T03:29:53.789ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2660 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 42d72d6c-462b-4482-b42a-3b1293e8dd92 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132482025-03-04T03:29:54.809ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 755c750d-1f0f-48a6-9805-0ceea14e5b17 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132492025-03-04T03:29:54.809ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 755c750d-1f0f-48a6-9805-0ceea14e5b17 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132502025-03-04T03:29:54.811ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2393 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 755c750d-1f0f-48a6-9805-0ceea14e5b17 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132512025-03-04T03:29:55.830ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 76989275-0927-4b96-9213-05789e57eb65 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132522025-03-04T03:29:55.830ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 76989275-0927-4b96-9213-05789e57eb65 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132532025-03-04T03:29:55.832ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1934 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 76989275-0927-4b96-9213-05789e57eb65 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132542025-03-04T03:29:56.851ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d2d3e809-046e-4435-856b-57da4023ed07 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132552025-03-04T03:29:56.852ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d2d3e809-046e-4435-856b-57da4023ed07 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132562025-03-04T03:29:56.853ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2460 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d2d3e809-046e-4435-856b-57da4023ed07 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132572025-03-04T03:29:57.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
132582025-03-04T03:29:57.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132592025-03-04T03:29:57.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132602025-03-04T03:29:57.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
132612025-03-04T03:29:57.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
132622025-03-04T03:29:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132632025-03-04T03:29:57.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132642025-03-04T03:29:57.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
132652025-03-04T03:29:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132662025-03-04T03:29:57.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
132672025-03-04T03:29:57.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132682025-03-04T03:29:57.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132692025-03-04T03:29:57.211ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.122851ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
132702025-03-04T03:29:57.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132712025-03-04T03:29:57.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132722025-03-04T03:29:57.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132732025-03-04T03:29:57.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
132742025-03-04T03:29:57.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132752025-03-04T03:29:57.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132762025-03-04T03:29:57.225ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
132772025-03-04T03:29:57.225ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.662201ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
132782025-03-04T03:29:57.872ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 027cfca9-0e80-4775-87f8-47501996afdf roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132792025-03-04T03:29:57.872ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 027cfca9-0e80-4775-87f8-47501996afdf resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132802025-03-04T03:29:57.874ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2490 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 027cfca9-0e80-4775-87f8-47501996afdf response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132812025-03-04T03:29:57.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 694 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 7ce9c328-7ab1-4dbd-801f-eac4901be33b response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
132822025-03-04T03:29:58.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e728e58-cfdb-4e93-9b4d-4d650c493cb5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132832025-03-04T03:29:58.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e728e58-cfdb-4e93-9b4d-4d650c493cb5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132842025-03-04T03:29:58.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2059 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2e728e58-cfdb-4e93-9b4d-4d650c493cb5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132852025-03-04T03:29:59.915ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef53038e-21a5-48a3-899f-6d8474436d6e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132862025-03-04T03:29:59.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef53038e-21a5-48a3-899f-6d8474436d6e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132872025-03-04T03:29:59.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2545 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ef53038e-21a5-48a3-899f-6d8474436d6e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132882025-03-04T03:30:00.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae54326b-3c4f-4af2-ac97-90fb7f5ad189 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132892025-03-04T03:30:00.936ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae54326b-3c4f-4af2-ac97-90fb7f5ad189 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132902025-03-04T03:30:00.938ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2372 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae54326b-3c4f-4af2-ac97-90fb7f5ad189 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132912025-03-04T03:30:01.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2b3718d1-d519-4746-8e8f-b762b0651418 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132922025-03-04T03:30:01.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2b3718d1-d519-4746-8e8f-b762b0651418 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132932025-03-04T03:30:01.960ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2737 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2b3718d1-d519-4746-8e8f-b762b0651418 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132942025-03-04T03:30:02.979ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48278ddd-f126-42c8-9474-5950c9de505e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132952025-03-04T03:30:02.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48278ddd-f126-42c8-9474-5950c9de505e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132962025-03-04T03:30:02.982ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2768 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48278ddd-f126-42c8-9474-5950c9de505e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132972025-03-04T03:30:04.001ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae440722-8b3d-4a3a-904d-623d4762d43c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132982025-03-04T03:30:04.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae440722-8b3d-4a3a-904d-623d4762d43c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
132992025-03-04T03:30:04.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2870 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ae440722-8b3d-4a3a-904d-623d4762d43c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133002025-03-04T03:30:05.024ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc28c402-c030-42d1-8f97-9f8b57043963 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133012025-03-04T03:30:05.025ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc28c402-c030-42d1-8f97-9f8b57043963 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133022025-03-04T03:30:05.026ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2202 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bc28c402-c030-42d1-8f97-9f8b57043963 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133032025-03-04T03:30:06.044ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 223078ff-845b-45bb-b77b-ba162b2947a9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133042025-03-04T03:30:06.045ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 223078ff-845b-45bb-b77b-ba162b2947a9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133052025-03-04T03:30:06.047ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2345 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 223078ff-845b-45bb-b77b-ba162b2947a9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133062025-03-04T03:30:07.066ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 598dd557-72cb-4d10-a653-4ae2fe3e8eee roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133072025-03-04T03:30:07.066ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 598dd557-72cb-4d10-a653-4ae2fe3e8eee resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133082025-03-04T03:30:07.068ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2152 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 598dd557-72cb-4d10-a653-4ae2fe3e8eee response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
133092025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
133102025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
133112025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
133122025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
133132025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
133142025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
133152025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
133162025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
133172025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
133182025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
133192025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
133202025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
133212025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
133222025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
133232025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
133242025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
133252025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
133262025-03-04T03:30:07.205ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:20:07.205434728 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
133272025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
133282025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
133292025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
133302025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
133312025-03-04T03:30:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
133322025-03-04T03:30:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 400368270 } })
133332025-03-04T03:30:07.205ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 400397891 } })
133342025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
133352025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
133362025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
133372025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133382025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
133392025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
133402025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
133412025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
133422025-03-04T03:30:07.206ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
133432025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
133442025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
133452025-03-04T03:30:07.206ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
133462025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
133472025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
133482025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.95\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
133492025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
133502025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
133512025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
133522025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
133532025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133542025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
133552025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
133562025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
133572025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
133582025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
133592025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
133602025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
133612025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
133622025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
133632025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
133642025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
133652025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133662025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
133672025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.246196ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
133682025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
133692025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
133702025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
133712025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
133722025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
133732025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
133742025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.271676ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
133752025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
133762025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
133772025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
133782025-03-04T03:30:07.206ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
133792025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
133802025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
133812025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
133822025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
133832025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
133842025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
133852025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
133862025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
133872025-03-04T03:30:07.207ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
133882025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133892025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
133902025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
133912025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
133922025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
133932025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
133942025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
133952025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133962025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
133972025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
133982025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
133992025-03-04T03:30:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
134002025-03-04T03:30:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
134012025-03-04T03:30:07.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 402573088 } })
134022025-03-04T03:30:07.208ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
134032025-03-04T03:30:07.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134042025-03-04T03:30:07.209ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
134052025-03-04T03:30:07.209ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
134062025-03-04T03:30:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
134072025-03-04T03:30:07.209ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 404169567 } })
134082025-03-04T03:30:07.209ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
134092025-03-04T03:30:07.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
134102025-03-04T03:30:07.210ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
134112025-03-04T03:30:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
134122025-03-04T03:30:07.210ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:20:07.205434728 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
134132025-03-04T03:30:07.210ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.375716ms iteration = 13 status = {"expiration":"2025-03-04T03:20:07.205434728Z","pruned":[]}
134142025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
134152025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
134162025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
134172025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
134182025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
134192025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
134202025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
134212025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
134222025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
134232025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
134242025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
134252025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
134262025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
134272025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
134282025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
134292025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
134302025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
134312025-03-04T03:30:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
134322025-03-04T03:30:07.212ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134332025-03-04T03:30:07.212ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.328828ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
134342025-03-04T03:30:07.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134352025-03-04T03:30:07.214ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134362025-03-04T03:30:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
134372025-03-04T03:30:07.214ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
134382025-03-04T03:30:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
134392025-03-04T03:30:07.215ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
134402025-03-04T03:30:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
134412025-03-04T03:30:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
134422025-03-04T03:30:07.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
134432025-03-04T03:30:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
134442025-03-04T03:30:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
134452025-03-04T03:30:07.216ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134462025-03-04T03:30:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134472025-03-04T03:30:07.216ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134482025-03-04T03:30:07.217ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134492025-03-04T03:30:07.217ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134502025-03-04T03:30:07.217ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134512025-03-04T03:30:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
134522025-03-04T03:30:07.220ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
134532025-03-04T03:30:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
134542025-03-04T03:30:07.220ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
134552025-03-04T03:30:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
134562025-03-04T03:30:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
134572025-03-04T03:30:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
134582025-03-04T03:30:07.223ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
134592025-03-04T03:30:07.223ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
134602025-03-04T03:30:07.223ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
134612025-03-04T03:30:07.223ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
134622025-03-04T03:30:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
134632025-03-04T03:30:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
134642025-03-04T03:30:07.224ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
134652025-03-04T03:30:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
134662025-03-04T03:30:07.224ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134672025-03-04T03:30:07.226ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134682025-03-04T03:30:07.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
134692025-03-04T03:30:07.226ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
134702025-03-04T03:30:07.227ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
134712025-03-04T03:30:07.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
134722025-03-04T03:30:07.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
134732025-03-04T03:30:07.228ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
134742025-03-04T03:30:07.228ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134752025-03-04T03:30:07.228ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134762025-03-04T03:30:07.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134772025-03-04T03:30:07.229ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
134782025-03-04T03:30:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
134792025-03-04T03:30:07.230ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134802025-03-04T03:30:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
134812025-03-04T03:30:07.230ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
134822025-03-04T03:30:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
134832025-03-04T03:30:07.232ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
134842025-03-04T03:30:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
134852025-03-04T03:30:07.233ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
134862025-03-04T03:30:07.233ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
134872025-03-04T03:30:07.233ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 24.031845ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
134882025-03-04T03:30:07.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
134892025-03-04T03:30:07.235ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 24.409679ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
134902025-03-04T03:30:07.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
134912025-03-04T03:30:07.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
134922025-03-04T03:30:07.240ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
134932025-03-04T03:30:07.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
134942025-03-04T03:30:07.240ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1719, tv_nsec: 434804103 } })
134952025-03-04T03:30:07.240ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
134962025-03-04T03:30:07.240ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
134972025-03-04T03:30:07.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
134982025-03-04T03:30:07.242ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134992025-03-04T03:30:07.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
135002025-03-04T03:30:07.245ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
135012025-03-04T03:30:07.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
135022025-03-04T03:30:07.247ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135032025-03-04T03:30:07.247ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 36.429176ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
135042025-03-04T03:30:07.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135052025-03-04T03:30:07.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
135062025-03-04T03:30:07.251ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
135072025-03-04T03:30:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
135082025-03-04T03:30:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
135092025-03-04T03:30:07.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135102025-03-04T03:30:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135112025-03-04T03:30:07.255ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135122025-03-04T03:30:07.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
135132025-03-04T03:30:07.257ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
135142025-03-04T03:30:07.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
135152025-03-04T03:30:07.257ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135162025-03-04T03:30:07.259ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135172025-03-04T03:30:07.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
135182025-03-04T03:30:07.260ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
135192025-03-04T03:30:07.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
135202025-03-04T03:30:07.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
135212025-03-04T03:30:07.262ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
135222025-03-04T03:30:07.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
135232025-03-04T03:30:07.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135242025-03-04T03:30:07.273ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 62.413165ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
135252025-03-04T03:30:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135262025-03-04T03:30:07.276ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 70.722677ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
135272025-03-04T03:30:07.277ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135282025-03-04T03:30:07.279ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
135292025-03-04T03:30:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
135302025-03-04T03:30:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
135312025-03-04T03:30:07.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
135322025-03-04T03:30:07.280ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
135332025-03-04T03:30:07.280ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
135342025-03-04T03:30:07.280ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 74.454843ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
135352025-03-04T03:30:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
135362025-03-04T03:30:07.281ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
135372025-03-04T03:30:07.281ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
135382025-03-04T03:30:07.281ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
135392025-03-04T03:30:07.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
135402025-03-04T03:30:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
135412025-03-04T03:30:07.283ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
135422025-03-04T03:30:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
135432025-03-04T03:30:07.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135442025-03-04T03:30:07.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 73.239517ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
135452025-03-04T03:30:07.285ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
135462025-03-04T03:30:07.285ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
135472025-03-04T03:30:07.285ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
135482025-03-04T03:30:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
135492025-03-04T03:30:07.286ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
135502025-03-04T03:30:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
135512025-03-04T03:30:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
135522025-03-04T03:30:07.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
135532025-03-04T03:30:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
135542025-03-04T03:30:07.289ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 78.157788ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
135552025-03-04T03:30:07.290ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 82.47637ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
135562025-03-04T03:30:07.301ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
135572025-03-04T03:30:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
135582025-03-04T03:30:07.303ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
135592025-03-04T03:30:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
135602025-03-04T03:30:07.344ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
135612025-03-04T03:30:07.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
135622025-03-04T03:30:07.362ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
135632025-03-04T03:30:07.362ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
135642025-03-04T03:30:07.362ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2daf5fef-893e-4710-9766-45538afc6875", "content-length": "26", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
135652025-03-04T03:30:07.362ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
135662025-03-04T03:30:07.364ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
135672025-03-04T03:30:07.364ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
135682025-03-04T03:30:07.365ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
135692025-03-04T03:30:07.365ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
135702025-03-04T03:30:07.365ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b321eb8-6192-4c86-8f52-4a01cb2dedff", "content-length": "26", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
135712025-03-04T03:30:07.365ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
135722025-03-04T03:30:07.367ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
135732025-03-04T03:30:07.424ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
135742025-03-04T03:30:07.424ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
135752025-03-04T03:30:07.424ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "882b3719-e900-4a80-a234-bb609818594a", "content-length": "26", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
135762025-03-04T03:30:07.424ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
135772025-03-04T03:30:07.426ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
135782025-03-04T03:30:07.458ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
135792025-03-04T03:30:07.557ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
135802025-03-04T03:30:07.588ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 382.982123ms iteration = 25 status = {}
135812025-03-04T03:30:07.598ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
135822025-03-04T03:30:07.598ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "f05aad7a-85d2-4827-afff-331855a4f7c4", "content-length": "2", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
135832025-03-04T03:30:07.599ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
135842025-03-04T03:30:07.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
135852025-03-04T03:30:07.601ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
135862025-03-04T03:30:07.601ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
135872025-03-04T03:30:07.602ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "21878382-a6a2-4ae6-9d34-2c6212ac40a8", "content-length": "372", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
135882025-03-04T03:30:07.602ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
135892025-03-04T03:30:07.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
135902025-03-04T03:30:07.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135912025-03-04T03:30:07.605ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 399.605308ms iteration = 6 status = {"nsuccess":5}
135922025-03-04T03:30:07.606ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135932025-03-04T03:30:07.606ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
135942025-03-04T03:30:07.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
135952025-03-04T03:30:07.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
135962025-03-04T03:30:07.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135972025-03-04T03:30:07.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135982025-03-04T03:30:07.608ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
135992025-03-04T03:30:07.608ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 400.667881ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
136002025-03-04T03:30:07.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
136012025-03-04T03:30:07.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
136022025-03-04T03:30:07.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
136032025-03-04T03:30:07.610ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136042025-03-04T03:30:07.611ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136052025-03-04T03:30:07.612ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136062025-03-04T03:30:07.612ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
136072025-03-04T03:30:07.612ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
136082025-03-04T03:30:07.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
136092025-03-04T03:30:07.612ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 407.574666ms iteration = 25 status = null
136102025-03-04T03:30:07.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
136112025-03-04T03:30:07.613ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136122025-03-04T03:30:07.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
136132025-03-04T03:30:07.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
136142025-03-04T03:30:07.614ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
136152025-03-04T03:30:07.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
136162025-03-04T03:30:07.618ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
136172025-03-04T03:30:07.618ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 412.546586ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
136182025-03-04T03:30:07.630ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
136192025-03-04T03:30:07.630ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
136202025-03-04T03:30:07.630ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0cc82b9c-4fde-4172-9bdd-edbf62c87520", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
136212025-03-04T03:30:07.630ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
136222025-03-04T03:30:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
136232025-03-04T03:30:07.637ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
136242025-03-04T03:30:07.638ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
136252025-03-04T03:30:07.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
136262025-03-04T03:30:07.638ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 431.944984ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
136272025-03-04T03:30:07.641ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
136282025-03-04T03:30:07.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
136292025-03-04T03:30:07.643ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
136302025-03-04T03:30:07.643ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
136312025-03-04T03:30:07.643ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
136322025-03-04T03:30:07.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
136332025-03-04T03:30:07.644ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
136342025-03-04T03:30:07.644ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
136352025-03-04T03:30:07.644ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1e2d913c-35e7-4cd0-ba6c-000d7d430ea0", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
136362025-03-04T03:30:07.645ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
136372025-03-04T03:30:07.645ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136382025-03-04T03:30:07.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
136392025-03-04T03:30:07.646ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 439.035771ms iteration = 26 status = {}
136402025-03-04T03:30:07.647ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
136412025-03-04T03:30:07.647ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
136422025-03-04T03:30:07.647ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
136432025-03-04T03:30:07.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
136442025-03-04T03:30:07.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
136452025-03-04T03:30:07.649ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
136462025-03-04T03:30:07.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
136472025-03-04T03:30:07.650ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 444.125283ms iteration = 28 status = {"physical_disks_added":0}
136482025-03-04T03:30:07.650ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
136492025-03-04T03:30:07.650ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "f893e4a3-b189-4b68-8667-ae03c309c30b", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
136502025-03-04T03:30:07.650ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
136512025-03-04T03:30:07.651ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
136522025-03-04T03:30:07.651ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
136532025-03-04T03:30:07.651ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
136542025-03-04T03:30:07.651ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 445.806333ms iteration = 13 status = {"generation":2}
136552025-03-04T03:30:07.652ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136562025-03-04T03:30:07.652ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e89d47f0-9606-4562-8420-4181ca871958", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
136572025-03-04T03:30:07.652ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
136582025-03-04T03:30:07.655ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
136592025-03-04T03:30:07.655ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 449.683412ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
136602025-03-04T03:30:07.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
136612025-03-04T03:30:07.655ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
136622025-03-04T03:30:07.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
136632025-03-04T03:30:07.656ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 450.722204ms iteration = 25 status = 0
136642025-03-04T03:30:07.660ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136652025-03-04T03:30:07.660ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
136662025-03-04T03:30:07.660ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
136672025-03-04T03:30:07.660ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
136682025-03-04T03:30:07.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
136692025-03-04T03:30:07.662ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
136702025-03-04T03:30:07.662ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136712025-03-04T03:30:07.662ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
136722025-03-04T03:30:07.662ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
136732025-03-04T03:30:07.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
136742025-03-04T03:30:07.662ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a9a1c415-bedc-4bf3-a2ed-116405ee5617", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
136752025-03-04T03:30:07.662ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
136762025-03-04T03:30:07.663ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "8d4a3c33-0227-4986-a68d-041467ac2634", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
136772025-03-04T03:30:07.663ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
136782025-03-04T03:30:07.664ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
136792025-03-04T03:30:07.664ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 459.178708ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
136802025-03-04T03:30:07.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
136812025-03-04T03:30:07.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
136822025-03-04T03:30:07.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
136832025-03-04T03:30:07.665ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "cc37aedf-d58a-447e-957e-0338d4844ca6", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
136842025-03-04T03:30:07.665ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
136852025-03-04T03:30:07.667ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
136862025-03-04T03:30:07.667ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
136872025-03-04T03:30:07.667ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
136882025-03-04T03:30:07.667ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 462.120203ms iteration = 13 status = {"generation":1}
136892025-03-04T03:30:07.667ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
136902025-03-04T03:30:07.667ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 462.261895ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
136912025-03-04T03:30:07.676ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 464.814407ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
136922025-03-04T03:30:07.679ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
136932025-03-04T03:30:07.679ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
136942025-03-04T03:30:07.679ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 471.870303ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
136952025-03-04T03:30:07.682ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
136962025-03-04T03:30:07.682ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 475.241104ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
136972025-03-04T03:30:07.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
136982025-03-04T03:30:07.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
136992025-03-04T03:30:07.720ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
137002025-03-04T03:30:07.720ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137012025-03-04T03:30:07.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137022025-03-04T03:30:07.728ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
137032025-03-04T03:30:07.728ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d01e75ea-5c81-47b4-af40-86e54de113ad", "content-length": "134", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
137042025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
137052025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
137062025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
137072025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
137082025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
137092025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
137102025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
137112025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
137122025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
137132025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
137142025-03-04T03:30:07.729ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
137152025-03-04T03:30:07.729ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137162025-03-04T03:30:07.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137172025-03-04T03:30:07.732ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
137182025-03-04T03:30:07.732ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d2d9555-72f8-4eee-98d5-88dba26069bb", "content-length": "2", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
137192025-03-04T03:30:07.732ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
137202025-03-04T03:30:07.732ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "448a5288-34c9-4bfc-ad21-9d7fc293454d", "content-length": "51", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
137212025-03-04T03:30:07.732ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
137222025-03-04T03:30:07.732ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
137232025-03-04T03:30:07.749ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
137242025-03-04T03:30:07.828ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "06e09822-ae57-4d97-9b93-32410bd29a7c", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
137252025-03-04T03:30:07.828ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
137262025-03-04T03:30:07.828ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
137272025-03-04T03:30:07.828ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e841aa1-26b7-4f3b-b3c1-d3544c594812", "content-length": "691", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
137282025-03-04T03:30:07.828ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
137292025-03-04T03:30:07.828ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
137302025-03-04T03:30:07.828ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
137312025-03-04T03:30:07.828ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "47ee00ff-0c6b-40ce-9546-1ee38c10129a", "content-length": "682", "date": "Tue, 04 Mar 2025 03:30:07 GMT"} })
137322025-03-04T03:30:07.828ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137332025-03-04T03:30:07.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137342025-03-04T03:30:07.830ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 624.17737ms iteration = 26 status = {}
137352025-03-04T03:30:07.832ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137362025-03-04T03:30:07.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137372025-03-04T03:30:07.836ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137382025-03-04T03:30:07.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137392025-03-04T03:30:07.838ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137402025-03-04T03:30:07.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137412025-03-04T03:30:07.846ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 640.168105ms iteration = 25 status = {}
137422025-03-04T03:30:07.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 523 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 2bb0f91b-26fb-4c67-b7d6-cabc6e66c53b response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
137432025-03-04T03:30:08.087ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 73f1d977-c786-45ff-9063-cd1a0a271d06 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137442025-03-04T03:30:08.087ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 73f1d977-c786-45ff-9063-cd1a0a271d06 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137452025-03-04T03:30:08.090ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 73f1d977-c786-45ff-9063-cd1a0a271d06 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137462025-03-04T03:30:09.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aa1a6aec-0aa9-4e93-af98-5d8895d30871 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137472025-03-04T03:30:09.111ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aa1a6aec-0aa9-4e93-af98-5d8895d30871 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137482025-03-04T03:30:09.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2545 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aa1a6aec-0aa9-4e93-af98-5d8895d30871 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137492025-03-04T03:30:10.131ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 285042d4-6761-4f98-b014-e7ee06a09943 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137502025-03-04T03:30:10.132ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 285042d4-6761-4f98-b014-e7ee06a09943 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137512025-03-04T03:30:10.134ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2823 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 285042d4-6761-4f98-b014-e7ee06a09943 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137522025-03-04T03:30:11.153ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d30bd02-c96c-41dd-84f5-f6d6b182ba1c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137532025-03-04T03:30:11.154ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d30bd02-c96c-41dd-84f5-f6d6b182ba1c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137542025-03-04T03:30:11.155ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2160 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1d30bd02-c96c-41dd-84f5-f6d6b182ba1c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137552025-03-04T03:30:12.175ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 24c9e4b3-5915-4e50-851a-1a2d3a806ce2 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137562025-03-04T03:30:12.175ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 24c9e4b3-5915-4e50-851a-1a2d3a806ce2 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137572025-03-04T03:30:12.177ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2764 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 24c9e4b3-5915-4e50-851a-1a2d3a806ce2 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137582025-03-04T03:30:13.197ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d143c23-7f08-42d4-b13f-fe5473fc2520 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137592025-03-04T03:30:13.197ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d143c23-7f08-42d4-b13f-fe5473fc2520 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137602025-03-04T03:30:13.199ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2467 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9d143c23-7f08-42d4-b13f-fe5473fc2520 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137612025-03-04T03:30:14.219ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a880f2b4-0ec1-45a6-9000-07a6c0512ba8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137622025-03-04T03:30:14.219ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a880f2b4-0ec1-45a6-9000-07a6c0512ba8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137632025-03-04T03:30:14.221ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2354 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a880f2b4-0ec1-45a6-9000-07a6c0512ba8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137642025-03-04T03:30:15.241ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc44b25e-f71f-488d-be92-0090274bc715 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137652025-03-04T03:30:15.241ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc44b25e-f71f-488d-be92-0090274bc715 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137662025-03-04T03:30:15.243ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2441 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = cc44b25e-f71f-488d-be92-0090274bc715 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137672025-03-04T03:30:16.262ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ac975c35-1aba-4c8d-9874-f66c2dd8ede4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137682025-03-04T03:30:16.263ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ac975c35-1aba-4c8d-9874-f66c2dd8ede4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137692025-03-04T03:30:16.264ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2040 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ac975c35-1aba-4c8d-9874-f66c2dd8ede4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137702025-03-04T03:30:17.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
137712025-03-04T03:30:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
137722025-03-04T03:30:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
137732025-03-04T03:30:17.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
137742025-03-04T03:30:17.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
137752025-03-04T03:30:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
137762025-03-04T03:30:17.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137772025-03-04T03:30:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
137782025-03-04T03:30:17.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
137792025-03-04T03:30:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
137802025-03-04T03:30:17.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
137812025-03-04T03:30:17.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
137822025-03-04T03:30:17.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137832025-03-04T03:30:17.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
137842025-03-04T03:30:17.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
137852025-03-04T03:30:17.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
137862025-03-04T03:30:17.228ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
137872025-03-04T03:30:17.228ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.600485ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
137882025-03-04T03:30:17.236ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
137892025-03-04T03:30:17.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
137902025-03-04T03:30:17.270ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.559769ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
137912025-03-04T03:30:17.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3a2233bd-c90f-4561-b070-6e834488e85f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137922025-03-04T03:30:17.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3a2233bd-c90f-4561-b070-6e834488e85f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137932025-03-04T03:30:17.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2313 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3a2233bd-c90f-4561-b070-6e834488e85f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137942025-03-04T03:30:17.897ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 511 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = a12f21c5-389f-4481-9647-1fc2caface92 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
137952025-03-04T03:30:18.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6c0ba04e-b2bb-4d2d-a0dc-2b6237c16d93 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137962025-03-04T03:30:18.305ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6c0ba04e-b2bb-4d2d-a0dc-2b6237c16d93 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137972025-03-04T03:30:18.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2178 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 6c0ba04e-b2bb-4d2d-a0dc-2b6237c16d93 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137982025-03-04T03:30:19.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a4b4dbde-7268-442d-9f6c-a48399328b0e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
137992025-03-04T03:30:19.326ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a4b4dbde-7268-442d-9f6c-a48399328b0e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138002025-03-04T03:30:19.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2030 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a4b4dbde-7268-442d-9f6c-a48399328b0e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138012025-03-04T03:30:20.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 931847fa-ce58-42e8-afd1-68035c106f5c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138022025-03-04T03:30:20.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 931847fa-ce58-42e8-afd1-68035c106f5c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138032025-03-04T03:30:20.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1999 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 931847fa-ce58-42e8-afd1-68035c106f5c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138042025-03-04T03:30:21.369ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7c2c0e9-6b97-4a62-94dd-4a29a237d4f7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138052025-03-04T03:30:21.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7c2c0e9-6b97-4a62-94dd-4a29a237d4f7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138062025-03-04T03:30:21.372ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2840 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b7c2c0e9-6b97-4a62-94dd-4a29a237d4f7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138072025-03-04T03:30:22.391ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d73f6d1f-9320-4207-b5bf-423e27d198d9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138082025-03-04T03:30:22.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d73f6d1f-9320-4207-b5bf-423e27d198d9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138092025-03-04T03:30:22.394ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2601 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d73f6d1f-9320-4207-b5bf-423e27d198d9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138102025-03-04T03:30:23.412ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f612be06-f236-4f3f-bfbc-1cfd468ab409 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138112025-03-04T03:30:23.413ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f612be06-f236-4f3f-bfbc-1cfd468ab409 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138122025-03-04T03:30:23.415ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2602 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f612be06-f236-4f3f-bfbc-1cfd468ab409 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138132025-03-04T03:30:24.434ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 839334dd-4bb4-44dc-9415-797a1c0ab9ec roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138142025-03-04T03:30:24.435ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 839334dd-4bb4-44dc-9415-797a1c0ab9ec resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138152025-03-04T03:30:24.437ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2753 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 839334dd-4bb4-44dc-9415-797a1c0ab9ec response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138162025-03-04T03:30:25.456ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 883f221e-66b0-4b6d-8541-09d5ada34e0d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138172025-03-04T03:30:25.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 883f221e-66b0-4b6d-8541-09d5ada34e0d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138182025-03-04T03:30:25.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1990 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 883f221e-66b0-4b6d-8541-09d5ada34e0d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138192025-03-04T03:30:26.477ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e033123c-9629-4253-8e07-ece70b714944 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138202025-03-04T03:30:26.478ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e033123c-9629-4253-8e07-ece70b714944 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138212025-03-04T03:30:26.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2143 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e033123c-9629-4253-8e07-ece70b714944 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138222025-03-04T03:30:27.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
138232025-03-04T03:30:27.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138242025-03-04T03:30:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138252025-03-04T03:30:27.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
138262025-03-04T03:30:27.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
138272025-03-04T03:30:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
138282025-03-04T03:30:27.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138292025-03-04T03:30:27.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
138302025-03-04T03:30:27.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138312025-03-04T03:30:27.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138322025-03-04T03:30:27.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138332025-03-04T03:30:27.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138342025-03-04T03:30:27.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138352025-03-04T03:30:27.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
138362025-03-04T03:30:27.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138372025-03-04T03:30:27.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138382025-03-04T03:30:27.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
138392025-03-04T03:30:27.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.334157ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
138402025-03-04T03:30:27.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
138412025-03-04T03:30:27.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
138422025-03-04T03:30:27.239ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.040552ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
138432025-03-04T03:30:27.498ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af505b18-7a75-4eaf-a1ca-8e39219bc952 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138442025-03-04T03:30:27.498ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af505b18-7a75-4eaf-a1ca-8e39219bc952 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138452025-03-04T03:30:27.500ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1861 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = af505b18-7a75-4eaf-a1ca-8e39219bc952 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138462025-03-04T03:30:27.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 501 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = dd3be619-ed5c-42fa-ab21-dd9b164fac74 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
138472025-03-04T03:30:28.519ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f8150d7-fcd0-439a-b37c-c95cecc24938 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138482025-03-04T03:30:28.519ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f8150d7-fcd0-439a-b37c-c95cecc24938 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138492025-03-04T03:30:28.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1976 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4f8150d7-fcd0-439a-b37c-c95cecc24938 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138502025-03-04T03:30:29.541ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eef29d1b-214f-4db1-9812-12551282ea5d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138512025-03-04T03:30:29.541ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eef29d1b-214f-4db1-9812-12551282ea5d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138522025-03-04T03:30:29.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2310 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eef29d1b-214f-4db1-9812-12551282ea5d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138532025-03-04T03:30:30.561ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f065fe96-60cd-4f0d-8e15-7ce5eebab6fd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138542025-03-04T03:30:30.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f065fe96-60cd-4f0d-8e15-7ce5eebab6fd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138552025-03-04T03:30:30.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2549 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f065fe96-60cd-4f0d-8e15-7ce5eebab6fd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138562025-03-04T03:30:31.582ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4a735c64-4a31-483c-989e-98dae628d777 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138572025-03-04T03:30:31.582ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4a735c64-4a31-483c-989e-98dae628d777 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138582025-03-04T03:30:31.584ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2444 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4a735c64-4a31-483c-989e-98dae628d777 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138592025-03-04T03:30:32.604ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4da79db2-0039-49e4-8935-450473467ccc roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138602025-03-04T03:30:32.604ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4da79db2-0039-49e4-8935-450473467ccc resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138612025-03-04T03:30:32.606ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2490 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4da79db2-0039-49e4-8935-450473467ccc response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138622025-03-04T03:30:33.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fe86411c-a487-42a0-9fe1-ea59d7fdd128 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138632025-03-04T03:30:33.626ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fe86411c-a487-42a0-9fe1-ea59d7fdd128 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138642025-03-04T03:30:33.628ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2465 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = fe86411c-a487-42a0-9fe1-ea59d7fdd128 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138652025-03-04T03:30:34.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 823e143f-f9fd-48dd-9f4f-6cd53f4dec98 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138662025-03-04T03:30:34.648ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 823e143f-f9fd-48dd-9f4f-6cd53f4dec98 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138672025-03-04T03:30:34.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2598 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 823e143f-f9fd-48dd-9f4f-6cd53f4dec98 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138682025-03-04T03:30:35.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7fbea667-713e-4237-9a73-8bdd3b7dac5e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138692025-03-04T03:30:35.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7fbea667-713e-4237-9a73-8bdd3b7dac5e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138702025-03-04T03:30:35.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2310 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7fbea667-713e-4237-9a73-8bdd3b7dac5e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138712025-03-04T03:30:36.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab2fe432-161d-467d-90b3-63ad3f37cfde roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138722025-03-04T03:30:36.690ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab2fe432-161d-467d-90b3-63ad3f37cfde resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138732025-03-04T03:30:36.692ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2413 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ab2fe432-161d-467d-90b3-63ad3f37cfde response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
138742025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
138752025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
138762025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
138772025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
138782025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
138792025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
138802025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
138812025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
138822025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
138832025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
138842025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
138852025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
138862025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138872025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
138882025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
138892025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
138902025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
138912025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138922025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
138932025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
138942025-03-04T03:30:37.206ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
138952025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138962025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
138972025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
138982025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
138992025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
139002025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
139012025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
139022025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
139032025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
139042025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
139052025-03-04T03:30:37.207ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
139062025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
139072025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
139082025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
139092025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
139102025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
139112025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
139122025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
139132025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
139142025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
139152025-03-04T03:30:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
139162025-03-04T03:30:37.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1749, tv_nsec: 402820104 } })
139172025-03-04T03:30:37.208ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
139182025-03-04T03:30:37.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139192025-03-04T03:30:37.209ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1749, tv_nsec: 403731365 } })
139202025-03-04T03:30:37.209ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
139212025-03-04T03:30:37.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139222025-03-04T03:30:37.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139232025-03-04T03:30:37.211ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.270182ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
139242025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
139252025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
139262025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
139272025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
139282025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
139292025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
139302025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
139312025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
139322025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
139332025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
139342025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
139352025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
139362025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
139372025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
139382025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
139392025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
139402025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
139412025-03-04T03:30:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
139422025-03-04T03:30:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
139432025-03-04T03:30:37.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139442025-03-04T03:30:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
139452025-03-04T03:30:37.213ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
139462025-03-04T03:30:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
139472025-03-04T03:30:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
139482025-03-04T03:30:37.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139492025-03-04T03:30:37.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139502025-03-04T03:30:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
139512025-03-04T03:30:37.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
139522025-03-04T03:30:37.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
139532025-03-04T03:30:37.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139542025-03-04T03:30:37.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139552025-03-04T03:30:37.216ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
139562025-03-04T03:30:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
139572025-03-04T03:30:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
139582025-03-04T03:30:37.217ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
139592025-03-04T03:30:37.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
139602025-03-04T03:30:37.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
139612025-03-04T03:30:37.218ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
139622025-03-04T03:30:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
139632025-03-04T03:30:37.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
139642025-03-04T03:30:37.219ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
139652025-03-04T03:30:37.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
139662025-03-04T03:30:37.222ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
139672025-03-04T03:30:37.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
139682025-03-04T03:30:37.223ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 11.628183ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
139692025-03-04T03:30:37.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139702025-03-04T03:30:37.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
139712025-03-04T03:30:37.226ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
139722025-03-04T03:30:37.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
139732025-03-04T03:30:37.226ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
139742025-03-04T03:30:37.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
139752025-03-04T03:30:37.227ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139762025-03-04T03:30:37.228ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1749, tv_nsec: 423118973 } })
139772025-03-04T03:30:37.229ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
139782025-03-04T03:30:37.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
139792025-03-04T03:30:37.229ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
139802025-03-04T03:30:37.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
139812025-03-04T03:30:37.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139822025-03-04T03:30:37.234ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
139832025-03-04T03:30:37.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139842025-03-04T03:30:37.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
139852025-03-04T03:30:37.247ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
139862025-03-04T03:30:37.247ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.679468ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
139872025-03-04T03:30:37.249ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
139882025-03-04T03:30:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
139892025-03-04T03:30:37.251ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
139902025-03-04T03:30:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
139912025-03-04T03:30:37.252ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 40.200833ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
139922025-03-04T03:30:37.254ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
139932025-03-04T03:30:37.254ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
139942025-03-04T03:30:37.254ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
139952025-03-04T03:30:37.254ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
139962025-03-04T03:30:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
139972025-03-04T03:30:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
139982025-03-04T03:30:37.256ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
139992025-03-04T03:30:37.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
140002025-03-04T03:30:37.258ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 51.583282ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
140012025-03-04T03:30:37.259ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140022025-03-04T03:30:37.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
140032025-03-04T03:30:37.262ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140042025-03-04T03:30:37.263ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 51.37584ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
140052025-03-04T03:30:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
140062025-03-04T03:30:37.264ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
140072025-03-04T03:30:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
140082025-03-04T03:30:37.264ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140092025-03-04T03:30:37.265ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
140102025-03-04T03:30:37.265ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 57.789608ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
140112025-03-04T03:30:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
140122025-03-04T03:30:37.266ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
140132025-03-04T03:30:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
140142025-03-04T03:30:37.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
140152025-03-04T03:30:37.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
140162025-03-04T03:30:37.356ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
140172025-03-04T03:30:37.356ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
140182025-03-04T03:30:37.356ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
140192025-03-04T03:30:37.356ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
140202025-03-04T03:30:37.356ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fae5748-dbd6-438d-bb1d-03517005b1ed", "content-length": "372", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
140212025-03-04T03:30:37.356ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
140222025-03-04T03:30:37.356ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4539a76f-3877-41bc-97f2-9812d0e12fa5", "content-length": "26", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
140232025-03-04T03:30:37.356ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
140242025-03-04T03:30:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
140252025-03-04T03:30:37.357ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
140262025-03-04T03:30:37.358ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
140272025-03-04T03:30:37.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
140282025-03-04T03:30:37.361ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140292025-03-04T03:30:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
140302025-03-04T03:30:37.364ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
140312025-03-04T03:30:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
140322025-03-04T03:30:37.368ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 161.946276ms iteration = 26 status = 0
140332025-03-04T03:30:37.396ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 189.906198ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
140342025-03-04T03:30:37.398ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 186.735318ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
140352025-03-04T03:30:37.401ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
140362025-03-04T03:30:37.401ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
140372025-03-04T03:30:37.401ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
140382025-03-04T03:30:37.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
140392025-03-04T03:30:37.403ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 191.443267ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
140402025-03-04T03:30:37.406ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
140412025-03-04T03:30:37.406ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 199.981981ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
140422025-03-04T03:30:37.410ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
140432025-03-04T03:30:37.410ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
140442025-03-04T03:30:37.410ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c93a034-64e4-4e38-a926-d0fc2bf677a9", "content-length": "26", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
140452025-03-04T03:30:37.410ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
140462025-03-04T03:30:37.412ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
140472025-03-04T03:30:37.413ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
140482025-03-04T03:30:37.413ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
140492025-03-04T03:30:37.413ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fa82e86e-074d-4df8-87e6-b8e6215ae328", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
140502025-03-04T03:30:37.413ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
140512025-03-04T03:30:37.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
140522025-03-04T03:30:37.415ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
140532025-03-04T03:30:37.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
140542025-03-04T03:30:37.419ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
140552025-03-04T03:30:37.419ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
140562025-03-04T03:30:37.419ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
140572025-03-04T03:30:37.419ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2142aeb0-b88d-41d7-ad28-70700c4c70b0", "content-length": "26", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
140582025-03-04T03:30:37.419ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
140592025-03-04T03:30:37.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
140602025-03-04T03:30:37.421ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
140612025-03-04T03:30:37.422ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
140622025-03-04T03:30:37.422ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
140632025-03-04T03:30:37.423ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7e693eda-cd9c-4275-9cf7-c9f1c840942c", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
140642025-03-04T03:30:37.424ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 217.400185ms iteration = 27 status = {}
140652025-03-04T03:30:37.441ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
140662025-03-04T03:30:37.441ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
140672025-03-04T03:30:37.441ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 234.052729ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
140682025-03-04T03:30:37.465ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140692025-03-04T03:30:37.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
140702025-03-04T03:30:37.466ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140712025-03-04T03:30:37.468ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
140722025-03-04T03:30:37.468ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140732025-03-04T03:30:37.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
140742025-03-04T03:30:37.470ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140752025-03-04T03:30:37.471ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
140762025-03-04T03:30:37.471ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140772025-03-04T03:30:37.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
140782025-03-04T03:30:37.473ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 266.03383ms iteration = 29 status = {"physical_disks_added":0}
140792025-03-04T03:30:37.651ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
140802025-03-04T03:30:37.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
140812025-03-04T03:30:37.653ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
140822025-03-04T03:30:37.653ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
140832025-03-04T03:30:37.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
140842025-03-04T03:30:37.662ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
140852025-03-04T03:30:37.662ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d3f97fe-5c24-4e6b-80dd-dae19178ac82", "content-length": "134", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
140862025-03-04T03:30:37.662ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
140872025-03-04T03:30:37.662ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
140882025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
140892025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
140902025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
140912025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
140922025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
140932025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
140942025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
140952025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
140962025-03-04T03:30:37.663ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
140972025-03-04T03:30:37.663ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
140982025-03-04T03:30:37.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
140992025-03-04T03:30:37.666ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
141002025-03-04T03:30:37.666ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "2753da43-3992-4626-aae9-c409a54a3236", "content-length": "2", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
141012025-03-04T03:30:37.666ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
141022025-03-04T03:30:37.666ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a308b3ee-7533-4ce0-919b-576eedc8d70a", "content-length": "51", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
141032025-03-04T03:30:37.666ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
141042025-03-04T03:30:37.666ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
141052025-03-04T03:30:37.670ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 463.916396ms iteration = 26 status = {}
141062025-03-04T03:30:37.674ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
141072025-03-04T03:30:37.674ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c8c0b1c-1d4a-4226-8efa-64354d6f97e0", "content-length": "2", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
141082025-03-04T03:30:37.676ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
141092025-03-04T03:30:37.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
141102025-03-04T03:30:37.678ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 472.215507ms iteration = 26 status = null
141112025-03-04T03:30:37.693ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
141122025-03-04T03:30:37.759ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d56dffc7-7d1a-4141-ac3c-c604b76b2460", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
141132025-03-04T03:30:37.759ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
141142025-03-04T03:30:37.759ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
141152025-03-04T03:30:37.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b08ba94e-acbe-407a-9d0a-09bea95caefd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141162025-03-04T03:30:37.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b08ba94e-acbe-407a-9d0a-09bea95caefd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141172025-03-04T03:30:37.760ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e47ec3f8-9ad5-414a-bd47-7818a493b2ec", "content-length": "691", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
141182025-03-04T03:30:37.760ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a033d0b-c17d-4ca9-963e-4119e1e301ad", "content-length": "682", "date": "Tue, 04 Mar 2025 03:30:37 GMT"} })
141192025-03-04T03:30:37.760ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
141202025-03-04T03:30:37.760ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
141212025-03-04T03:30:37.760ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
141222025-03-04T03:30:37.760ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
141232025-03-04T03:30:37.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
141242025-03-04T03:30:37.762ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 555.562588ms iteration = 27 status = {}
141252025-03-04T03:30:37.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2615 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b08ba94e-acbe-407a-9d0a-09bea95caefd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141262025-03-04T03:30:37.764ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
141272025-03-04T03:30:37.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
141282025-03-04T03:30:37.767ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
141292025-03-04T03:30:37.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
141302025-03-04T03:30:37.770ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
141312025-03-04T03:30:37.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
141322025-03-04T03:30:37.777ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 570.958157ms iteration = 26 status = {}
141332025-03-04T03:30:37.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 494 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = bffa1da9-071e-4927-a163-95c66c1c74d3 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
141342025-03-04T03:30:38.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7426168-020b-46ad-93af-d4bc9bc0b25b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141352025-03-04T03:30:38.783ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7426168-020b-46ad-93af-d4bc9bc0b25b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141362025-03-04T03:30:38.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 7403 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d7426168-020b-46ad-93af-d4bc9bc0b25b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141372025-03-04T03:30:39.815ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 584d1d00-03be-44f5-b83b-fffcb7f0f0f0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141382025-03-04T03:30:39.815ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 584d1d00-03be-44f5-b83b-fffcb7f0f0f0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141392025-03-04T03:30:39.817ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2737 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 584d1d00-03be-44f5-b83b-fffcb7f0f0f0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141402025-03-04T03:30:40.837ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eaf286c0-505d-410f-a532-5d3be24816d1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141412025-03-04T03:30:40.838ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eaf286c0-505d-410f-a532-5d3be24816d1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141422025-03-04T03:30:40.840ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2798 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = eaf286c0-505d-410f-a532-5d3be24816d1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141432025-03-04T03:30:41.859ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 689ea498-d97f-4daa-93c3-e1ff53121bac roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141442025-03-04T03:30:41.859ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 689ea498-d97f-4daa-93c3-e1ff53121bac resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141452025-03-04T03:30:41.861ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2458 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 689ea498-d97f-4daa-93c3-e1ff53121bac response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141462025-03-04T03:30:42.881ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9fe3cab7-9abc-4c46-b405-6240b739d9d6 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141472025-03-04T03:30:42.881ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9fe3cab7-9abc-4c46-b405-6240b739d9d6 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141482025-03-04T03:30:42.884ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2896 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9fe3cab7-9abc-4c46-b405-6240b739d9d6 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141492025-03-04T03:30:43.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bf46eca2-24ce-46c4-b822-12a675cfd3fe roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141502025-03-04T03:30:43.903ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bf46eca2-24ce-46c4-b822-12a675cfd3fe resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141512025-03-04T03:30:43.905ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2389 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bf46eca2-24ce-46c4-b822-12a675cfd3fe response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141522025-03-04T03:30:44.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f0feeeeb-d9bb-45b1-9167-3abfa0101235 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141532025-03-04T03:30:44.925ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f0feeeeb-d9bb-45b1-9167-3abfa0101235 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141542025-03-04T03:30:44.927ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2453 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f0feeeeb-d9bb-45b1-9167-3abfa0101235 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141552025-03-04T03:30:45.947ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8045217e-a236-4d6a-b1ee-c33a9d896780 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141562025-03-04T03:30:45.947ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8045217e-a236-4d6a-b1ee-c33a9d896780 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141572025-03-04T03:30:45.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2260 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8045217e-a236-4d6a-b1ee-c33a9d896780 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141582025-03-04T03:30:46.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 02e309ef-9918-4cb2-99bc-e4769e532109 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141592025-03-04T03:30:46.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 02e309ef-9918-4cb2-99bc-e4769e532109 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141602025-03-04T03:30:46.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2048 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 02e309ef-9918-4cb2-99bc-e4769e532109 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141612025-03-04T03:30:47.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
141622025-03-04T03:30:47.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
141632025-03-04T03:30:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
141642025-03-04T03:30:47.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
141652025-03-04T03:30:47.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
141662025-03-04T03:30:47.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
141672025-03-04T03:30:47.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141682025-03-04T03:30:47.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
141692025-03-04T03:30:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
141702025-03-04T03:30:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
141712025-03-04T03:30:47.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
141722025-03-04T03:30:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
141732025-03-04T03:30:47.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
141742025-03-04T03:30:47.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
141752025-03-04T03:30:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141762025-03-04T03:30:47.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
141772025-03-04T03:30:47.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
141782025-03-04T03:30:47.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
141792025-03-04T03:30:47.227ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
141802025-03-04T03:30:47.227ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.277111ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
141812025-03-04T03:30:47.238ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.549276ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
141822025-03-04T03:30:47.892ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:52081
141832025-03-04T03:30:47.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52081 req_id = ee25eaf9-632e-46e3-8f2f-6b9d3a089fca roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
141842025-03-04T03:30:47.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52081 req_id = ee25eaf9-632e-46e3-8f2f-6b9d3a089fca resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
141852025-03-04T03:30:47.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3296 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52081 req_id = ee25eaf9-632e-46e3-8f2f-6b9d3a089fca response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
141862025-03-04T03:30:47.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52081 req_id = eb2a18aa-1023-4bbf-a6ce-509022063fb5 roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
141872025-03-04T03:30:47.896ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52081 req_id = eb2a18aa-1023-4bbf-a6ce-509022063fb5 resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
141882025-03-04T03:30:47.898ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2394 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:52081 req_id = eb2a18aa-1023-4bbf-a6ce-509022063fb5 response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
141892025-03-04T03:30:47.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 663 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 0d26393f-d2f0-4483-ad23-bcaa5cf4f9ee response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
141902025-03-04T03:30:47.991ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec520e9b-d5d1-4022-ad12-8e3216b27811 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141912025-03-04T03:30:47.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec520e9b-d5d1-4022-ad12-8e3216b27811 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141922025-03-04T03:30:47.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2070 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ec520e9b-d5d1-4022-ad12-8e3216b27811 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141932025-03-04T03:30:49.012ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9958781-5001-4029-9e45-4f23639c6dbb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141942025-03-04T03:30:49.012ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9958781-5001-4029-9e45-4f23639c6dbb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141952025-03-04T03:30:49.014ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1736 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d9958781-5001-4029-9e45-4f23639c6dbb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141962025-03-04T03:30:50.034ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 65e6bfb0-ab84-48f3-862b-1f07365f4e49 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141972025-03-04T03:30:50.034ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 65e6bfb0-ab84-48f3-862b-1f07365f4e49 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141982025-03-04T03:30:50.036ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2569 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 65e6bfb0-ab84-48f3-862b-1f07365f4e49 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
141992025-03-04T03:30:51.054ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edcd2fa9-c06f-4fbb-972a-a2deaaa1f757 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142002025-03-04T03:30:51.055ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edcd2fa9-c06f-4fbb-972a-a2deaaa1f757 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142012025-03-04T03:30:51.057ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2641 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = edcd2fa9-c06f-4fbb-972a-a2deaaa1f757 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142022025-03-04T03:30:52.076ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e0fb98a-376e-4cd6-aacf-fb78455e62c1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142032025-03-04T03:30:52.077ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e0fb98a-376e-4cd6-aacf-fb78455e62c1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142042025-03-04T03:30:52.079ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2545 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e0fb98a-376e-4cd6-aacf-fb78455e62c1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142052025-03-04T03:30:53.100ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c77ba1f7-925a-415c-bbe8-d501efd292bf roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142062025-03-04T03:30:53.100ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c77ba1f7-925a-415c-bbe8-d501efd292bf resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142072025-03-04T03:30:53.102ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2625 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c77ba1f7-925a-415c-bbe8-d501efd292bf response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142082025-03-04T03:30:54.122ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 85fcbe27-5eb8-4fe3-bcfc-8b58c62ec1f5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142092025-03-04T03:30:54.123ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 85fcbe27-5eb8-4fe3-bcfc-8b58c62ec1f5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142102025-03-04T03:30:54.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2333 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 85fcbe27-5eb8-4fe3-bcfc-8b58c62ec1f5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142112025-03-04T03:30:55.144ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 341796e8-04da-4fe8-b938-710f3439008e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142122025-03-04T03:30:55.144ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 341796e8-04da-4fe8-b938-710f3439008e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142132025-03-04T03:30:55.146ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2699 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 341796e8-04da-4fe8-b938-710f3439008e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142142025-03-04T03:30:56.166ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de2b77af-e4c3-4193-a0a7-5f0ae5be1c97 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142152025-03-04T03:30:56.166ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de2b77af-e4c3-4193-a0a7-5f0ae5be1c97 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142162025-03-04T03:30:56.169ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2714 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = de2b77af-e4c3-4193-a0a7-5f0ae5be1c97 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142172025-03-04T03:30:57.187ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 35061d2f-7850-4306-aa95-aa2aaa4f9020 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142182025-03-04T03:30:57.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 35061d2f-7850-4306-aa95-aa2aaa4f9020 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142192025-03-04T03:30:57.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2221 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 35061d2f-7850-4306-aa95-aa2aaa4f9020 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142202025-03-04T03:30:57.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
142212025-03-04T03:30:57.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
142222025-03-04T03:30:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
142232025-03-04T03:30:57.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
142242025-03-04T03:30:57.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
142252025-03-04T03:30:57.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
142262025-03-04T03:30:57.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
142272025-03-04T03:30:57.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
142282025-03-04T03:30:57.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142292025-03-04T03:30:57.240ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
142302025-03-04T03:30:57.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
142312025-03-04T03:30:57.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
142322025-03-04T03:30:57.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
142332025-03-04T03:30:57.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
142342025-03-04T03:30:57.245ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.614641ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
142352025-03-04T03:30:57.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142362025-03-04T03:30:57.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
142372025-03-04T03:30:57.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
142382025-03-04T03:30:57.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
142392025-03-04T03:30:57.288ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
142402025-03-04T03:30:57.288ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 81.729332ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
142412025-03-04T03:30:57.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 661 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 335b1c68-6136-4de7-8869-0b3113ed1d19 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
142422025-03-04T03:30:58.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9da6d9e6-b39f-4d31-84a1-3a2c2d3d98d1 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142432025-03-04T03:30:58.209ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9da6d9e6-b39f-4d31-84a1-3a2c2d3d98d1 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142442025-03-04T03:30:58.211ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2634 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9da6d9e6-b39f-4d31-84a1-3a2c2d3d98d1 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142452025-03-04T03:30:59.229ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f2049875-43ad-45ef-b2bf-740b5c1e434a roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142462025-03-04T03:30:59.230ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f2049875-43ad-45ef-b2bf-740b5c1e434a resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142472025-03-04T03:30:59.231ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f2049875-43ad-45ef-b2bf-740b5c1e434a response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142482025-03-04T03:31:00.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51469dc7-1cff-46ca-91c8-cfbd02b796aa roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142492025-03-04T03:31:00.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51469dc7-1cff-46ca-91c8-cfbd02b796aa resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142502025-03-04T03:31:00.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2091 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 51469dc7-1cff-46ca-91c8-cfbd02b796aa response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142512025-03-04T03:31:01.272ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a3529db-1a0b-4621-af2d-74b883205f6e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142522025-03-04T03:31:01.273ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a3529db-1a0b-4621-af2d-74b883205f6e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142532025-03-04T03:31:01.275ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2610 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0a3529db-1a0b-4621-af2d-74b883205f6e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142542025-03-04T03:31:02.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4dbabefd-a226-4b0c-951a-8fcaa89e23a9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142552025-03-04T03:31:02.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4dbabefd-a226-4b0c-951a-8fcaa89e23a9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142562025-03-04T03:31:02.297ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2508 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 4dbabefd-a226-4b0c-951a-8fcaa89e23a9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142572025-03-04T03:31:03.317ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c9c5557f-4eb9-460c-9d8e-c02e97e3faf5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142582025-03-04T03:31:03.317ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c9c5557f-4eb9-460c-9d8e-c02e97e3faf5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142592025-03-04T03:31:03.319ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2007 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c9c5557f-4eb9-460c-9d8e-c02e97e3faf5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142602025-03-04T03:31:04.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 663351e6-55db-40a3-8233-cd8cf9849b8f roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142612025-03-04T03:31:04.340ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 663351e6-55db-40a3-8233-cd8cf9849b8f resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142622025-03-04T03:31:04.342ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2693 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 663351e6-55db-40a3-8233-cd8cf9849b8f response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142632025-03-04T03:31:05.361ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 037385a1-a294-4749-b1d7-6973f79b6bb7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142642025-03-04T03:31:05.362ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 037385a1-a294-4749-b1d7-6973f79b6bb7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142652025-03-04T03:31:05.364ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2420 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 037385a1-a294-4749-b1d7-6973f79b6bb7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142662025-03-04T03:31:06.383ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 403b16cb-fb96-4b82-8031-f786216b1daf roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142672025-03-04T03:31:06.384ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 403b16cb-fb96-4b82-8031-f786216b1daf resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142682025-03-04T03:31:06.386ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2450 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 403b16cb-fb96-4b82-8031-f786216b1daf response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
142692025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
142702025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
142712025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
142722025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
142732025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
142742025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
142752025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
142762025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
142772025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
142782025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
142792025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
142802025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
142812025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
142822025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
142832025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
142842025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
142852025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
142862025-03-04T03:31:07.206ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-04 03:21:07.206440562 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
142872025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
142882025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
142892025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
142902025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
142912025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
142922025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
142932025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
142942025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
142952025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
142962025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
142972025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
142982025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
142992025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
143002025-03-04T03:31:07.206ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce
143012025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
143022025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
143032025-03-04T03:31:07.206ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
143042025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
143052025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 22.911\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce"}
143062025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
143072025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
143082025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
143092025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
143102025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
143112025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
143122025-03-04T03:31:07.206ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e2dad65-6466-48b5-bc3a-8a9f5178c9b7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("72700cb3-c938-46f0-9b9c-46cf389d975e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 400950683 } })
143132025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
143142025-03-04T03:31:07.206ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("13b4e778-1c69-4f32-9182-d8fdea8251cd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99798c93-1ef2-44fb-9fde-7dd780a62a5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a5620ef8-a00b-4ddd-8732-aebedbdd8941.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 400971304 } })
143152025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
143162025-03-04T03:31:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
143172025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143182025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
143192025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
143202025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
143212025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
143222025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
143232025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
143242025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
143252025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
143262025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.078254ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
143272025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
143282025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
143292025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
143302025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
143312025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
143322025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
143332025-03-04T03:31:07.207ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
143342025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.096504ms iteration = 14 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
143352025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
143362025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
143372025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
143382025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
143392025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
143402025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
143412025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
143422025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
143432025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
143442025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
143452025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
143462025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
143472025-03-04T03:31:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
143482025-03-04T03:31:07.209ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
143492025-03-04T03:31:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
143502025-03-04T03:31:07.209ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
143512025-03-04T03:31:07.209ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
143522025-03-04T03:31:07.210ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 404072472 } })
143532025-03-04T03:31:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
143542025-03-04T03:31:07.210ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
143552025-03-04T03:31:07.210ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 404793931 } })
143562025-03-04T03:31:07.211ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
143572025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.608206ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
143582025-03-04T03:31:07.211ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-04 03:21:07.206440562 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
143592025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.994591ms iteration = 14 status = {"expiration":"2025-03-04T03:21:07.206440562Z","pruned":[]}
143602025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143612025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143622025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
143632025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
143642025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
143652025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
143662025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
143672025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
143682025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
143692025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
143702025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
143712025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
143722025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
143732025-03-04T03:31:07.211ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
143742025-03-04T03:31:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
143752025-03-04T03:31:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
143762025-03-04T03:31:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
143772025-03-04T03:31:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
143782025-03-04T03:31:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
143792025-03-04T03:31:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
143802025-03-04T03:31:07.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143812025-03-04T03:31:07.214ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
143822025-03-04T03:31:07.214ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
143832025-03-04T03:31:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
143842025-03-04T03:31:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
143852025-03-04T03:31:07.214ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143862025-03-04T03:31:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143872025-03-04T03:31:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
143882025-03-04T03:31:07.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
143892025-03-04T03:31:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
143902025-03-04T03:31:07.216ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143912025-03-04T03:31:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
143922025-03-04T03:31:07.217ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
143932025-03-04T03:31:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
143942025-03-04T03:31:07.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143952025-03-04T03:31:07.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143962025-03-04T03:31:07.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
143972025-03-04T03:31:07.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
143982025-03-04T03:31:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
143992025-03-04T03:31:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
144002025-03-04T03:31:07.220ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
144012025-03-04T03:31:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
144022025-03-04T03:31:07.222ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144032025-03-04T03:31:07.222ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144042025-03-04T03:31:07.226ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.575854ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
144052025-03-04T03:31:07.226ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 14.973893ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
144062025-03-04T03:31:07.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
144072025-03-04T03:31:07.227ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
144082025-03-04T03:31:07.231ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
144092025-03-04T03:31:07.231ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
144102025-03-04T03:31:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
144112025-03-04T03:31:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
144122025-03-04T03:31:07.232ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1779, tv_nsec: 426889181 } })
144132025-03-04T03:31:07.233ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
144142025-03-04T03:31:07.234ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144152025-03-04T03:31:07.236ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
144162025-03-04T03:31:07.236ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
144172025-03-04T03:31:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
144182025-03-04T03:31:07.239ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
144192025-03-04T03:31:07.239ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 32.60272ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
144202025-03-04T03:31:07.257ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
144212025-03-04T03:31:07.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
144222025-03-04T03:31:07.260ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
144232025-03-04T03:31:07.260ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
144242025-03-04T03:31:07.260ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
144252025-03-04T03:31:07.260ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
144262025-03-04T03:31:07.260ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 53.570166ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
144272025-03-04T03:31:07.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
144282025-03-04T03:31:07.265ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
144292025-03-04T03:31:07.265ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
144302025-03-04T03:31:07.265ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 55.555031ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
144312025-03-04T03:31:07.266ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144322025-03-04T03:31:07.267ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
144332025-03-04T03:31:07.267ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144342025-03-04T03:31:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
144352025-03-04T03:31:07.267ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144362025-03-04T03:31:07.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144372025-03-04T03:31:07.268ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
144382025-03-04T03:31:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
144392025-03-04T03:31:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
144402025-03-04T03:31:07.270ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
144412025-03-04T03:31:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
144422025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
144432025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
144442025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
144452025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
144462025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
144472025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
144482025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
144492025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
144502025-03-04T03:31:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
144512025-03-04T03:31:07.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.817005ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
144522025-03-04T03:31:07.273ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.392332ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
144532025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
144542025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 0 version = 2
144552025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
144562025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
144572025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
144582025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
144592025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
144602025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
144612025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
144622025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
144632025-03-04T03:31:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
144642025-03-04T03:31:07.275ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
144652025-03-04T03:31:07.275ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
144662025-03-04T03:31:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
144672025-03-04T03:31:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
144682025-03-04T03:31:07.275ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
144692025-03-04T03:31:07.275ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144702025-03-04T03:31:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
144712025-03-04T03:31:07.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144722025-03-04T03:31:07.276ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
144732025-03-04T03:31:07.276ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
144742025-03-04T03:31:07.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
144752025-03-04T03:31:07.276ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 64.806275ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
144762025-03-04T03:31:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
144772025-03-04T03:31:07.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144782025-03-04T03:31:07.278ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
144792025-03-04T03:31:07.278ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
144802025-03-04T03:31:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
144812025-03-04T03:31:07.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144822025-03-04T03:31:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
144832025-03-04T03:31:07.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
144842025-03-04T03:31:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
144852025-03-04T03:31:07.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144862025-03-04T03:31:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
144872025-03-04T03:31:07.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
144882025-03-04T03:31:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
144892025-03-04T03:31:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
144902025-03-04T03:31:07.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
144912025-03-04T03:31:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
144922025-03-04T03:31:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
144932025-03-04T03:31:07.282ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
144942025-03-04T03:31:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
144952025-03-04T03:31:07.284ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
144962025-03-04T03:31:07.284ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
144972025-03-04T03:31:07.284ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
144982025-03-04T03:31:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
144992025-03-04T03:31:07.284ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = bb946ac2-79ae-4525-813e-f3d61cf0a756 dns_zone_name = oxide.test found_so_far = 1 version = 2
145002025-03-04T03:31:07.284ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
145012025-03-04T03:31:07.284ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-04 03:18:37.943412 UTC dns_group = external generation = 2
145022025-03-04T03:31:07.284ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 78.169558ms iteration = 14 status = {"generation":2}
145032025-03-04T03:31:07.284ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145042025-03-04T03:31:07.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
145052025-03-04T03:31:07.288ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
145062025-03-04T03:31:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
145072025-03-04T03:31:07.290ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
145082025-03-04T03:31:07.290ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
145092025-03-04T03:31:07.290ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 84.230153ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
145102025-03-04T03:31:07.298ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
145112025-03-04T03:31:07.298ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 90.644351ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
145122025-03-04T03:31:07.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
145132025-03-04T03:31:07.310ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
145142025-03-04T03:31:07.311ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 104.377369ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
145152025-03-04T03:31:07.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
145162025-03-04T03:31:07.311ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
145172025-03-04T03:31:07.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
145182025-03-04T03:31:07.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 101.632106ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
145192025-03-04T03:31:07.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
145202025-03-04T03:31:07.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
145212025-03-04T03:31:07.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
145222025-03-04T03:31:07.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
145232025-03-04T03:31:07.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145242025-03-04T03:31:07.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
145252025-03-04T03:31:07.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
145262025-03-04T03:31:07.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
145272025-03-04T03:31:07.321ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.470445ms iteration = 27 status = 0
145282025-03-04T03:31:07.329ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
145292025-03-04T03:31:07.330ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
145302025-03-04T03:31:07.330ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
145312025-03-04T03:31:07.330ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
145322025-03-04T03:31:07.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
145332025-03-04T03:31:07.331ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
145342025-03-04T03:31:07.331ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
145352025-03-04T03:31:07.331ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 124.413074ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
145362025-03-04T03:31:07.332ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145372025-03-04T03:31:07.333ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
145382025-03-04T03:31:07.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
145392025-03-04T03:31:07.334ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145402025-03-04T03:31:07.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
145412025-03-04T03:31:07.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
145422025-03-04T03:31:07.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
145432025-03-04T03:31:07.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
145442025-03-04T03:31:07.336ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
145452025-03-04T03:31:07.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
145462025-03-04T03:31:07.337ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 130.936485ms iteration = 30 status = {"physical_disks_added":0}
145472025-03-04T03:31:07.337ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 0847ca34-8aab-4c16-be22-73028a498fab dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
145482025-03-04T03:31:07.337ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
145492025-03-04T03:31:07.337ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-04 03:18:36.932954 UTC dns_group = internal generation = 1
145502025-03-04T03:31:07.337ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 131.306849ms iteration = 14 status = {"generation":1}
145512025-03-04T03:31:07.343ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
145522025-03-04T03:31:07.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
145532025-03-04T03:31:07.349ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 137.560096ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
145542025-03-04T03:31:07.355ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
145552025-03-04T03:31:07.355ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
145562025-03-04T03:31:07.356ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e422ab0-c2ce-4fe7-901d-d8dca33c989b", "content-length": "26", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
145572025-03-04T03:31:07.356ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
145582025-03-04T03:31:07.358ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
145592025-03-04T03:31:07.376ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-04T03:18:03.594194Z, time_modified: 2025-03-04T03:18:03.594194Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, db_silo: Silo { identity: SiloIdentity { id: 99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-04T03:18:37.862442Z, time_modified: 2025-03-04T03:18:37.862442Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8" }] }) }
145602025-03-04T03:31:07.376ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 169.77988ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]}},"default_endpoint":{"silo_id":"99ae007e-ba78-4e30-b5ae-c2c9f3ed8d8b","tls_certs":["691ad1e917c120a16a4ba1637ed828c71e73e9e71f765db3375e1d0307f365b8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
145612025-03-04T03:31:07.394ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
145622025-03-04T03:31:07.394ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
145632025-03-04T03:31:07.394ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6955742a-74ef-4284-8a54-87b72fcc2248", "content-length": "26", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
145642025-03-04T03:31:07.394ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
145652025-03-04T03:31:07.396ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
145662025-03-04T03:31:07.404ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
145672025-03-04T03:31:07.404ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 197.318819ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
145682025-03-04T03:31:07.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 612b1623-8b38-4427-943c-34964ee95991 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
145692025-03-04T03:31:07.406ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 612b1623-8b38-4427-943c-34964ee95991 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
145702025-03-04T03:31:07.440ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
145712025-03-04T03:31:07.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
145722025-03-04T03:31:07.462ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
145732025-03-04T03:31:07.462ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
145742025-03-04T03:31:07.462ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7ae2170-425d-474b-a273-82e268fc3164", "content-length": "26", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
145752025-03-04T03:31:07.462ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
145762025-03-04T03:31:07.464ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
145772025-03-04T03:31:07.481ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
145782025-03-04T03:31:07.491ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
145792025-03-04T03:31:07.543ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
145802025-03-04T03:31:07.544ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "01411fb1-f2a2-4f7b-9a7d-79b7b5be950a", "content-length": "2", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
145812025-03-04T03:31:07.545ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
145822025-03-04T03:31:07.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
145832025-03-04T03:31:07.549ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
145842025-03-04T03:31:07.549ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
145852025-03-04T03:31:07.549ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "56ea4a8e-b110-47f2-80d9-e53a8b15b09e", "content-length": "372", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
145862025-03-04T03:31:07.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
145872025-03-04T03:31:07.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
145882025-03-04T03:31:07.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 145747 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 612b1623-8b38-4427-943c-34964ee95991 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
145892025-03-04T03:31:07.552ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 345.901179ms iteration = 27 status = null
145902025-03-04T03:31:07.577ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145912025-03-04T03:31:07.578ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
145922025-03-04T03:31:07.578ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
145932025-03-04T03:31:07.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
145942025-03-04T03:31:07.592ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
145952025-03-04T03:31:07.592ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
145962025-03-04T03:31:07.592ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d9a35062-ffe2-4224-a96b-dc474d684461", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
145972025-03-04T03:31:07.592ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
145982025-03-04T03:31:07.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
145992025-03-04T03:31:07.595ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
146002025-03-04T03:31:07.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
146012025-03-04T03:31:07.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
146022025-03-04T03:31:07.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
146032025-03-04T03:31:07.600ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
146042025-03-04T03:31:07.600ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
146052025-03-04T03:31:07.601ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7911550a-c65c-4fd1-8eb4-5185f46158c6", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
146062025-03-04T03:31:07.602ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 394.824819ms iteration = 28 status = {}
146072025-03-04T03:31:07.606ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146082025-03-04T03:31:07.608ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
146092025-03-04T03:31:07.608ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
146102025-03-04T03:31:07.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
146112025-03-04T03:31:07.619ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
146122025-03-04T03:31:07.619ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "49c0f20b-0be9-4de9-8399-40f4a950e4e3", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
146132025-03-04T03:31:07.619ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
146142025-03-04T03:31:07.621ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3014e9af-70e8-425d-aefe-23d21b512727", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
146152025-03-04T03:31:07.621ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
146162025-03-04T03:31:07.623ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "0da57715-4c1d-438a-8b7f-32bcdab6e792", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
146172025-03-04T03:31:07.623ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
146182025-03-04T03:31:07.624ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
146192025-03-04T03:31:07.624ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 418.154755ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
146202025-03-04T03:31:07.725ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 519.398415ms iteration = 27 status = {}
146212025-03-04T03:31:07.725ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
146222025-03-04T03:31:07.726ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "444f2ee0-bf9b-4b41-8b7a-de5ac014deec", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
146232025-03-04T03:31:07.726ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
146242025-03-04T03:31:07.728ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c1009c26-a285-4a34-9dec-793d3e7e1c88", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
146252025-03-04T03:31:07.728ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
146262025-03-04T03:31:07.729ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
146272025-03-04T03:31:07.729ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 523.388345ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
146282025-03-04T03:31:07.745ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
146292025-03-04T03:31:07.745ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "743c7142-8c60-4f8b-8530-baef0104f4df", "content-length": "691", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
146302025-03-04T03:31:07.745ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
146312025-03-04T03:31:07.745ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
146322025-03-04T03:31:07.745ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
146332025-03-04T03:31:07.746ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 540.020919ms iteration = 28 status = {}
146342025-03-04T03:31:07.750ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
146352025-03-04T03:31:07.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
146362025-03-04T03:31:07.753ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
146372025-03-04T03:31:07.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
146382025-03-04T03:31:07.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
146392025-03-04T03:31:07.761ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
146402025-03-04T03:31:07.762ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "12c0616f-ab61-41eb-91b7-9a946d20a0e9", "content-length": "134", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
146412025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
146422025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
146432025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
146442025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
146452025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
146462025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
146472025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
146482025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
146492025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
146502025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
146512025-03-04T03:31:07.762ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
146522025-03-04T03:31:07.762ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
146532025-03-04T03:31:07.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
146542025-03-04T03:31:07.764ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
146552025-03-04T03:31:07.764ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "69324fe8-9672-4995-8072-60c0e29fe3f4", "content-length": "2", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
146562025-03-04T03:31:07.764ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
146572025-03-04T03:31:07.765ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4e2b466-3053-47da-a0ee-b5e1b8efb903", "content-length": "51", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
146582025-03-04T03:31:07.765ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
146592025-03-04T03:31:07.765ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
146602025-03-04T03:31:07.857ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e48481e5-de6b-4902-a1db-2e3123afaade", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
146612025-03-04T03:31:07.857ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
146622025-03-04T03:31:07.857ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
146632025-03-04T03:31:07.857ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd2967f2-d7d8-44a5-b5af-b8a7e55a2b17", "content-length": "682", "date": "Tue, 04 Mar 2025 03:31:07 GMT"} })
146642025-03-04T03:31:07.857ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
146652025-03-04T03:31:07.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
146662025-03-04T03:31:07.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
146672025-03-04T03:31:07.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
146682025-03-04T03:31:07.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
146692025-03-04T03:31:07.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
146702025-03-04T03:31:07.866ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
146712025-03-04T03:31:07.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
146722025-03-04T03:31:07.873ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 667.208517ms iteration = 27 status = {}
146732025-03-04T03:31:07.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 497 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = e5e0aa0e-f896-4ac6-8123-5764c1ad4ea5 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
146742025-03-04T03:31:08.572ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 602e3659-ad84-4d31-9a64-97b68a160faa roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146752025-03-04T03:31:08.572ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 602e3659-ad84-4d31-9a64-97b68a160faa resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146762025-03-04T03:31:08.574ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2587 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 602e3659-ad84-4d31-9a64-97b68a160faa response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146772025-03-04T03:31:09.594ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2ba408e7-6909-45f2-b398-288abb8bbff9 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146782025-03-04T03:31:09.594ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2ba408e7-6909-45f2-b398-288abb8bbff9 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146792025-03-04T03:31:09.596ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2468 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 2ba408e7-6909-45f2-b398-288abb8bbff9 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146802025-03-04T03:31:10.615ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ea89cfa1-060f-487b-8b4d-2729cf6ed2b4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146812025-03-04T03:31:10.616ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ea89cfa1-060f-487b-8b4d-2729cf6ed2b4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146822025-03-04T03:31:10.617ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2199 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = ea89cfa1-060f-487b-8b4d-2729cf6ed2b4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146832025-03-04T03:31:11.637ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9fa8b704-8b07-47cd-99f6-db6d349a47d8 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146842025-03-04T03:31:11.637ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9fa8b704-8b07-47cd-99f6-db6d349a47d8 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146852025-03-04T03:31:11.640ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3037 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9fa8b704-8b07-47cd-99f6-db6d349a47d8 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146862025-03-04T03:31:12.659ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 922802b8-60eb-42c0-b3fc-766a1d76b17c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146872025-03-04T03:31:12.659ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 922802b8-60eb-42c0-b3fc-766a1d76b17c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146882025-03-04T03:31:12.662ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2932 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 922802b8-60eb-42c0-b3fc-766a1d76b17c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146892025-03-04T03:31:13.681ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21bd7b45-bfda-442b-bf88-244f00fb8cb7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146902025-03-04T03:31:13.681ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21bd7b45-bfda-442b-bf88-244f00fb8cb7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146912025-03-04T03:31:13.683ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2649 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 21bd7b45-bfda-442b-bf88-244f00fb8cb7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146922025-03-04T03:31:14.704ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e99e4a7-b746-4380-b2c1-6da44e709dd7 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146932025-03-04T03:31:14.705ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e99e4a7-b746-4380-b2c1-6da44e709dd7 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146942025-03-04T03:31:14.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2078 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 1e99e4a7-b746-4380-b2c1-6da44e709dd7 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146952025-03-04T03:31:15.726ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08e03000-17ff-4069-b9b7-b412350bc250 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146962025-03-04T03:31:15.726ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08e03000-17ff-4069-b9b7-b412350bc250 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146972025-03-04T03:31:15.728ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2607 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 08e03000-17ff-4069-b9b7-b412350bc250 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146982025-03-04T03:31:16.748ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 34e99fbd-9ef7-4c19-8061-6afbe8715b75 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
146992025-03-04T03:31:16.749ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 34e99fbd-9ef7-4c19-8061-6afbe8715b75 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147002025-03-04T03:31:16.751ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2588 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 34e99fbd-9ef7-4c19-8061-6afbe8715b75 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147012025-03-04T03:31:17.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
147022025-03-04T03:31:17.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147032025-03-04T03:31:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
147042025-03-04T03:31:17.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
147052025-03-04T03:31:17.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
147062025-03-04T03:31:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
147072025-03-04T03:31:17.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147082025-03-04T03:31:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
147092025-03-04T03:31:17.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147102025-03-04T03:31:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
147112025-03-04T03:31:17.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147122025-03-04T03:31:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
147132025-03-04T03:31:17.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
147142025-03-04T03:31:17.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
147152025-03-04T03:31:17.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147162025-03-04T03:31:17.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
147172025-03-04T03:31:17.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147182025-03-04T03:31:17.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
147192025-03-04T03:31:17.258ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
147202025-03-04T03:31:17.258ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 51.874065ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
147212025-03-04T03:31:17.265ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.772228ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
147222025-03-04T03:31:17.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6e41d80-0a9d-44b8-92f9-3e306bce3f43 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147232025-03-04T03:31:17.772ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6e41d80-0a9d-44b8-92f9-3e306bce3f43 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147242025-03-04T03:31:17.774ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2673 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d6e41d80-0a9d-44b8-92f9-3e306bce3f43 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147252025-03-04T03:31:17.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 604 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 7c7b842c-368c-4ebc-b084-721716e82a98 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
147262025-03-04T03:31:18.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 70de4257-1f0c-42dc-b946-399863efe9eb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147272025-03-04T03:31:18.794ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 70de4257-1f0c-42dc-b946-399863efe9eb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147282025-03-04T03:31:18.796ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2084 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 70de4257-1f0c-42dc-b946-399863efe9eb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147292025-03-04T03:31:19.817ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 828bbd7e-2967-4506-8954-09de86fefaf0 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147302025-03-04T03:31:19.817ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 828bbd7e-2967-4506-8954-09de86fefaf0 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147312025-03-04T03:31:19.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2268 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 828bbd7e-2967-4506-8954-09de86fefaf0 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147322025-03-04T03:31:20.838ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04adc3e9-620e-437e-ba9d-0e5dd09f692c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147332025-03-04T03:31:20.838ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04adc3e9-620e-437e-ba9d-0e5dd09f692c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147342025-03-04T03:31:20.840ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2479 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 04adc3e9-620e-437e-ba9d-0e5dd09f692c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147352025-03-04T03:31:21.859ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aa4eab92-b961-41c6-8bd7-d1ac51034615 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147362025-03-04T03:31:21.859ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aa4eab92-b961-41c6-8bd7-d1ac51034615 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147372025-03-04T03:31:21.862ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2805 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = aa4eab92-b961-41c6-8bd7-d1ac51034615 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147382025-03-04T03:31:22.882ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b2325cf9-e7a8-4e70-919b-eab585388c7e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147392025-03-04T03:31:22.882ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b2325cf9-e7a8-4e70-919b-eab585388c7e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147402025-03-04T03:31:22.884ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2633 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b2325cf9-e7a8-4e70-919b-eab585388c7e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147412025-03-04T03:31:23.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bbc9561b-46bd-43c6-b69c-f1ea0f6d13eb roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147422025-03-04T03:31:23.903ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bbc9561b-46bd-43c6-b69c-f1ea0f6d13eb resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147432025-03-04T03:31:23.905ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2036 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = bbc9561b-46bd-43c6-b69c-f1ea0f6d13eb response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147442025-03-04T03:31:24.923ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 49d5a8d6-08e5-43d7-bcdc-a1039e47dd0d roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147452025-03-04T03:31:24.924ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 49d5a8d6-08e5-43d7-bcdc-a1039e47dd0d resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147462025-03-04T03:31:24.926ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2422 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 49d5a8d6-08e5-43d7-bcdc-a1039e47dd0d response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147472025-03-04T03:31:25.945ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1629d49-68b8-4951-b454-1f46c7be598e roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147482025-03-04T03:31:25.946ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1629d49-68b8-4951-b454-1f46c7be598e resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147492025-03-04T03:31:25.948ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2625 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d1629d49-68b8-4951-b454-1f46c7be598e response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147502025-03-04T03:31:26.967ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9ed01891-55c9-4133-aac4-e6d08d599948 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147512025-03-04T03:31:26.968ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9ed01891-55c9-4133-aac4-e6d08d599948 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147522025-03-04T03:31:26.970ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2353 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9ed01891-55c9-4133-aac4-e6d08d599948 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147532025-03-04T03:31:27.207ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
147542025-03-04T03:31:27.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147552025-03-04T03:31:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
147562025-03-04T03:31:27.207ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
147572025-03-04T03:31:27.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
147582025-03-04T03:31:27.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
147592025-03-04T03:31:27.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147602025-03-04T03:31:27.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
147612025-03-04T03:31:27.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
147622025-03-04T03:31:27.239ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
147632025-03-04T03:31:27.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147642025-03-04T03:31:27.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
147652025-03-04T03:31:27.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147662025-03-04T03:31:27.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
147672025-03-04T03:31:27.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147682025-03-04T03:31:27.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
147692025-03-04T03:31:27.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147702025-03-04T03:31:27.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
147712025-03-04T03:31:27.257ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
147722025-03-04T03:31:27.257ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.485589ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
147732025-03-04T03:31:27.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.240336ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
147742025-03-04T03:31:27.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 517 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 1a9533f7-bb05-4d92-a2f2-44224d444cc4 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
147752025-03-04T03:31:27.990ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a61b8e12-aac8-4b03-82e6-cf40f8580994 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147762025-03-04T03:31:27.990ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a61b8e12-aac8-4b03-82e6-cf40f8580994 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147772025-03-04T03:31:27.991ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1926 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a61b8e12-aac8-4b03-82e6-cf40f8580994 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147782025-03-04T03:31:29.011ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 240c67e5-e93d-4302-9426-67d25f81eff4 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147792025-03-04T03:31:29.011ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 240c67e5-e93d-4302-9426-67d25f81eff4 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147802025-03-04T03:31:29.013ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2275 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 240c67e5-e93d-4302-9426-67d25f81eff4 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147812025-03-04T03:31:30.033ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f2d3c794-7f60-4867-8bc0-bba41305fdcd roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147822025-03-04T03:31:30.033ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f2d3c794-7f60-4867-8bc0-bba41305fdcd resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147832025-03-04T03:31:30.035ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2536 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = f2d3c794-7f60-4867-8bc0-bba41305fdcd response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147842025-03-04T03:31:31.055ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3623b321-0eef-42e3-bd58-b915fa4c7a45 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147852025-03-04T03:31:31.056ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3623b321-0eef-42e3-bd58-b915fa4c7a45 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147862025-03-04T03:31:31.058ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2775 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3623b321-0eef-42e3-bd58-b915fa4c7a45 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147872025-03-04T03:31:32.077ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a0ac702e-a7b9-4810-8b64-6e5931216493 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147882025-03-04T03:31:32.078ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a0ac702e-a7b9-4810-8b64-6e5931216493 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147892025-03-04T03:31:32.079ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2276 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a0ac702e-a7b9-4810-8b64-6e5931216493 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147902025-03-04T03:31:33.100ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b0a5d5a-5460-4cb3-85c3-9847cb9f78f5 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147912025-03-04T03:31:33.100ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b0a5d5a-5460-4cb3-85c3-9847cb9f78f5 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147922025-03-04T03:31:33.102ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2134 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 9b0a5d5a-5460-4cb3-85c3-9847cb9f78f5 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147932025-03-04T03:31:34.121ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a43b630b-ea6e-47e2-bd2a-f84e835ca878 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147942025-03-04T03:31:34.121ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a43b630b-ea6e-47e2-bd2a-f84e835ca878 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147952025-03-04T03:31:34.123ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2282 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = a43b630b-ea6e-47e2-bd2a-f84e835ca878 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147962025-03-04T03:31:35.143ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c0d2311f-19a4-4246-ae90-a93e5ebd6908 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147972025-03-04T03:31:35.144ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c0d2311f-19a4-4246-ae90-a93e5ebd6908 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147982025-03-04T03:31:35.146ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2646 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = c0d2311f-19a4-4246-ae90-a93e5ebd6908 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
147992025-03-04T03:31:36.165ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 02443f6c-2e8b-49c6-a3f5-6b84210f5a94 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
148002025-03-04T03:31:36.166ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 02443f6c-2e8b-49c6-a3f5-6b84210f5a94 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
148012025-03-04T03:31:36.168ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2308 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 02443f6c-2e8b-49c6-a3f5-6b84210f5a94 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
148022025-03-04T03:31:37.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0d1250f9-fbb7-4c73-ba61-23009d199217 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
148032025-03-04T03:31:37.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0d1250f9-fbb7-4c73-ba61-23009d199217 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
148042025-03-04T03:31:37.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2240 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 0d1250f9-fbb7-4c73-ba61-23009d199217 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
148052025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
148062025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
148072025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
148082025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
148092025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
148102025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
148112025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
148122025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
148132025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
148142025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
148152025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
148162025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
148172025-03-04T03:31:37.206ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
148182025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
148192025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148202025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
148212025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
148222025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
148232025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
148242025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
148252025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
148262025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
148272025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
148282025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
148292025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
148302025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
148312025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
148322025-03-04T03:31:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
148332025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
148342025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
148352025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
148362025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
148372025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
148382025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
148392025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
148402025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
148412025-03-04T03:31:37.208ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
148422025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
148432025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
148442025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
148452025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
148462025-03-04T03:31:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
148472025-03-04T03:31:37.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1809, tv_nsec: 402464255 } })
148482025-03-04T03:31:37.208ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1809, tv_nsec: 402594507 } })
148492025-03-04T03:31:37.208ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1155
148502025-03-04T03:31:37.209ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1155
148512025-03-04T03:31:37.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148522025-03-04T03:31:37.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148532025-03-04T03:31:37.210ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.817467ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
148542025-03-04T03:31:37.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148552025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
148562025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
148572025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
148582025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
148592025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
148602025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
148612025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
148622025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
148632025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
148642025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
148652025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
148662025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
148672025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
148682025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
148692025-03-04T03:31:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
148702025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
148712025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
148722025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
148732025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
148742025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148752025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
148762025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
148772025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
148782025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
148792025-03-04T03:31:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
148802025-03-04T03:31:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
148812025-03-04T03:31:37.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148822025-03-04T03:31:37.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148832025-03-04T03:31:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
148842025-03-04T03:31:37.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
148852025-03-04T03:31:37.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
148862025-03-04T03:31:37.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
148872025-03-04T03:31:37.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
148882025-03-04T03:31:37.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
148892025-03-04T03:31:37.223ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
148902025-03-04T03:31:37.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148912025-03-04T03:31:37.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
148922025-03-04T03:31:37.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
148932025-03-04T03:31:37.224ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
148942025-03-04T03:31:37.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
148952025-03-04T03:31:37.260ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
148962025-03-04T03:31:37.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
148972025-03-04T03:31:37.260ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 48.748008ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
148982025-03-04T03:31:37.262ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
148992025-03-04T03:31:37.262ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
149002025-03-04T03:31:37.262ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
149012025-03-04T03:31:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
149022025-03-04T03:31:37.263ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149032025-03-04T03:31:37.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
149042025-03-04T03:31:37.264ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
149052025-03-04T03:31:37.264ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149062025-03-04T03:31:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
149072025-03-04T03:31:37.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 59.298237ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
149082025-03-04T03:31:37.267ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 55.193687ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
149092025-03-04T03:31:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
149102025-03-04T03:31:37.268ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
149112025-03-04T03:31:37.268ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
149122025-03-04T03:31:37.268ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
149132025-03-04T03:31:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
149142025-03-04T03:31:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
149152025-03-04T03:31:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
149162025-03-04T03:31:37.269ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 57.12163ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
149172025-03-04T03:31:37.269ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
149182025-03-04T03:31:37.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
149192025-03-04T03:31:37.272ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
149202025-03-04T03:31:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
149212025-03-04T03:31:37.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149222025-03-04T03:31:37.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149232025-03-04T03:31:37.277ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
149242025-03-04T03:31:37.277ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 68.950545ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
149252025-03-04T03:31:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
149262025-03-04T03:31:37.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
149272025-03-04T03:31:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
149282025-03-04T03:31:37.278ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149292025-03-04T03:31:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
149302025-03-04T03:31:37.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149312025-03-04T03:31:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
149322025-03-04T03:31:37.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
149332025-03-04T03:31:37.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
149342025-03-04T03:31:37.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
149352025-03-04T03:31:37.280ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
149362025-03-04T03:31:37.280ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
149372025-03-04T03:31:37.280ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5525e426-7299-43ab-ad2b-346968b24e1c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1809, tv_nsec: 474722211 } })
149382025-03-04T03:31:37.281ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "864e7f58-ffaa-45ce-bce3-3fe0950f0818", "content-length": "26", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
149392025-03-04T03:31:37.281ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
149402025-03-04T03:31:37.281ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1155 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
149412025-03-04T03:31:37.282ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
149422025-03-04T03:31:37.286ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1163 zone_address = fd00:1122:3344:101::2
149432025-03-04T03:31:37.286ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
149442025-03-04T03:31:37.287ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0221fc9-c31e-4a68-ad57-fd753f60c825", "content-length": "26", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
149452025-03-04T03:31:37.287ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1168 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
149462025-03-04T03:31:37.288ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
149472025-03-04T03:31:37.291ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
149482025-03-04T03:31:37.291ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 84.081001ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
149492025-03-04T03:31:37.293ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 84.994911ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
149502025-03-04T03:31:37.294ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
149512025-03-04T03:31:37.294ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
149522025-03-04T03:31:37.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
149532025-03-04T03:31:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
149542025-03-04T03:31:37.297ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
149552025-03-04T03:31:37.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
149562025-03-04T03:31:37.301ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
149572025-03-04T03:31:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
149582025-03-04T03:31:37.302ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149592025-03-04T03:31:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
149602025-03-04T03:31:37.305ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
149612025-03-04T03:31:37.305ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 93.389015ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
149622025-03-04T03:31:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
149632025-03-04T03:31:37.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149642025-03-04T03:31:37.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5525e426-7299-43ab-ad2b-346968b24e1c, lookup_type: ById(5525e426-7299-43ab-ad2b-346968b24e1c) } result = Ok(())
149652025-03-04T03:31:37.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
149662025-03-04T03:31:37.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
149672025-03-04T03:31:37.323ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 116.166433ms iteration = 28 status = 0
149682025-03-04T03:31:37.333ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 121.182876ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
149692025-03-04T03:31:37.334ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
149702025-03-04T03:31:37.334ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
149712025-03-04T03:31:37.334ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 126.233127ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
149722025-03-04T03:31:37.337ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
149732025-03-04T03:31:37.337ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 130.290517ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
149742025-03-04T03:31:37.341ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
149752025-03-04T03:31:37.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
149762025-03-04T03:31:37.350ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
149772025-03-04T03:31:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
149782025-03-04T03:31:37.362ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
149792025-03-04T03:31:37.362ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7d71045-bc84-4200-9991-bfad09394e7e", "content-length": "2", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
149802025-03-04T03:31:37.364ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
149812025-03-04T03:31:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
149822025-03-04T03:31:37.366ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 160.218683ms iteration = 28 status = null
149832025-03-04T03:31:37.371ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 zone_address = fd00:1122:3344:101::2
149842025-03-04T03:31:37.371ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
149852025-03-04T03:31:37.371ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "80a1f971-ed7b-46b6-8064-899fa9cb0946", "content-length": "26", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
149862025-03-04T03:31:37.371ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1168 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
149872025-03-04T03:31:37.374ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1203 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
149882025-03-04T03:31:37.379ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 173.512996ms iteration = 28 status = {}
149892025-03-04T03:31:37.380ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149902025-03-04T03:31:37.382ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
149912025-03-04T03:31:37.382ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
149922025-03-04T03:31:37.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
149932025-03-04T03:31:37.391ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 184.232598ms iteration = 31 status = {"physical_disks_added":0}
149942025-03-04T03:31:37.412ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149952025-03-04T03:31:37.414ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
149962025-03-04T03:31:37.414ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
149972025-03-04T03:31:37.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
149982025-03-04T03:31:37.549ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
149992025-03-04T03:31:37.549ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
150002025-03-04T03:31:37.550ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "54fe2d2d-6175-405b-bc0c-af7de7028a83", "content-length": "372", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150012025-03-04T03:31:37.550ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150022025-03-04T03:31:37.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150032025-03-04T03:31:37.590ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
150042025-03-04T03:31:37.590ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
150052025-03-04T03:31:37.590ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "33fc5f8e-b398-4b2d-9483-9d096ab4a6f8", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150062025-03-04T03:31:37.590ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150072025-03-04T03:31:37.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150082025-03-04T03:31:37.593ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150092025-03-04T03:31:37.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150102025-03-04T03:31:37.596ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150112025-03-04T03:31:37.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150122025-03-04T03:31:37.599ZINFOnexus (BackgroundTasks): VPC route manager sled 5525e426-7299-43ab-ad2b-346968b24e1c push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
150132025-03-04T03:31:37.600ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
150142025-03-04T03:31:37.600ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ded43466-762e-4b6d-a36c-813dce15f24d", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150152025-03-04T03:31:37.602ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 394.260492ms iteration = 29 status = {}
150162025-03-04T03:31:37.619ZDEBGnexus (BackgroundTasks): client request SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
150172025-03-04T03:31:37.619ZDEBGnexus (BackgroundTasks): client response SledAgent = 5525e426-7299-43ab-ad2b-346968b24e1c background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ae72e1f-2011-4064-86af-d3d7bf11ccc6", "content-length": "691", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150182025-03-04T03:31:37.619ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
150192025-03-04T03:31:37.619ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
150202025-03-04T03:31:37.619ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
150212025-03-04T03:31:37.620ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 413.847532ms iteration = 29 status = {}
150222025-03-04T03:31:37.669ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150232025-03-04T03:31:37.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150242025-03-04T03:31:37.672ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6444a949-7236-4f2a-a163-c861165757f7, rack_id: accfd3a5-958c-4905-8c5c-5f2ac43dda05, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(720fab27-ce90-4441-b11f-d98932cca1bc) } rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
150252025-03-04T03:31:37.672ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150262025-03-04T03:31:37.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150272025-03-04T03:31:37.681ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
150282025-03-04T03:31:37.681ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f6d9b43-db6b-4808-ae91-38b58b5c99da", "content-length": "134", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150292025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
150302025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 720fab27-ce90-4441-b11f-d98932cca1bc
150312025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
150322025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
150332025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
150342025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
150352025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
150362025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
150372025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
150382025-03-04T03:31:37.681ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
150392025-03-04T03:31:37.682ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05
150402025-03-04T03:31:37.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150412025-03-04T03:31:37.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150422025-03-04T03:31:37.683ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
150432025-03-04T03:31:37.683ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "96d5bc40-f1d3-4284-a85c-390135ff17b7", "content-length": "2", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150442025-03-04T03:31:37.683ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
150452025-03-04T03:31:37.683ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "71f73efe-7f0f-4896-859b-d06e08d84286", "content-length": "51", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150462025-03-04T03:31:37.684ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch_location = Switch0
150472025-03-04T03:31:37.684ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
150482025-03-04T03:31:37.774ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3ec3bb20-715a-4819-a6bc-352f643e16c5", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150492025-03-04T03:31:37.774ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 switch = Switch0
150502025-03-04T03:31:37.774ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
150512025-03-04T03:31:37.774ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = accfd3a5-958c-4905-8c5c-5f2ac43dda05 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4d8e60a-14b5-4269-b69f-8f8140927301", "content-length": "682", "date": "Tue, 04 Mar 2025 03:31:37 GMT"} })
150522025-03-04T03:31:37.774ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150532025-03-04T03:31:37.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150542025-03-04T03:31:37.778ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150552025-03-04T03:31:37.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150562025-03-04T03:31:37.782ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150572025-03-04T03:31:37.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150582025-03-04T03:31:37.784ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150592025-03-04T03:31:37.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150602025-03-04T03:31:37.792ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 585.086801ms iteration = 28 status = {}
150612025-03-04T03:31:37.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 501 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 08457928-84da-4c10-bb6d-73b13d03fbe6 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
150622025-03-04T03:31:38.064ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
150632025-03-04T03:31:38.064ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
150642025-03-04T03:31:38.095ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
150652025-03-04T03:31:38.095ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:57302
150662025-03-04T03:31:38.095ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:57302 req_id = 67147583-d2fa-43f2-ac82-d9e6c92ca54b roles = RoleSet { roles: {} } uri = /metrics/producers
150672025-03-04T03:31:38.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:57302 req_id = 67147583-d2fa-43f2-ac82-d9e6c92ca54b resource = Database result = Ok(()) uri = /metrics/producers
150682025-03-04T03:31:38.133ZINFOnexus: assigned collector to new producer collector_id = ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a file = nexus/src/app/oximeter.rs:76 producer_id = c28b34ab-1b39-4325-87ad-c4ccf5b009d5
150692025-03-04T03:31:38.133ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 37634 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:57302 req_id = 67147583-d2fa-43f2-ac82-d9e6c92ca54b response_code = 201 uri = /metrics/producers
150702025-03-04T03:31:38.133ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "67147583-d2fa-43f2-ac82-d9e6c92ca54b", "content-length": "41", "date": "Tue, 04 Mar 2025 03:31:38 GMT"} })
150712025-03-04T03:31:38.135ZDEBGnexus (producer-registration-task): registered with nexus successfully
150722025-03-04T03:31:38.135ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
150732025-03-04T03:31:38.207ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48196ada-0ead-4bf8-9acd-a9a28a3ece11 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150742025-03-04T03:31:38.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48196ada-0ead-4bf8-9acd-a9a28a3ece11 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150752025-03-04T03:31:38.210ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2398 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 48196ada-0ead-4bf8-9acd-a9a28a3ece11 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150762025-03-04T03:31:39.299ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09a14b74-2904-4184-a301-f47c3b981668 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150772025-03-04T03:31:39.299ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09a14b74-2904-4184-a301-f47c3b981668 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150782025-03-04T03:31:39.300ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1952 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 09a14b74-2904-4184-a301-f47c3b981668 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150792025-03-04T03:31:40.320ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d02107d6-d244-4a18-80f7-0baab38bd9ad roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150802025-03-04T03:31:40.321ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d02107d6-d244-4a18-80f7-0baab38bd9ad resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150812025-03-04T03:31:40.323ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2516 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = d02107d6-d244-4a18-80f7-0baab38bd9ad response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150822025-03-04T03:31:41.342ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3ee81dc0-bea7-4de4-a0f8-97a456f61a86 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150832025-03-04T03:31:41.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3ee81dc0-bea7-4de4-a0f8-97a456f61a86 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150842025-03-04T03:31:41.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3030 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 3ee81dc0-bea7-4de4-a0f8-97a456f61a86 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150852025-03-04T03:31:42.365ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 29209eb1-1792-4d80-87bc-f33b5b76e6ce roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150862025-03-04T03:31:42.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 29209eb1-1792-4d80-87bc-f33b5b76e6ce resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150872025-03-04T03:31:42.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3179 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 29209eb1-1792-4d80-87bc-f33b5b76e6ce response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150882025-03-04T03:31:43.389ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b22fc203-82a9-4c98-ac3a-2520af8ffa07 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150892025-03-04T03:31:43.390ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b22fc203-82a9-4c98-ac3a-2520af8ffa07 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150902025-03-04T03:31:43.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2849 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = b22fc203-82a9-4c98-ac3a-2520af8ffa07 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150912025-03-04T03:31:44.413ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dc9b840c-119e-48b6-96b0-639c47c82f5b roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150922025-03-04T03:31:44.413ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dc9b840c-119e-48b6-96b0-639c47c82f5b resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150932025-03-04T03:31:44.415ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2471 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = dc9b840c-119e-48b6-96b0-639c47c82f5b response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150942025-03-04T03:31:45.433ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0c85859-d982-42ad-88a5-fccfc794f4ea roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150952025-03-04T03:31:45.434ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0c85859-d982-42ad-88a5-fccfc794f4ea resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150962025-03-04T03:31:45.436ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3021 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = e0c85859-d982-42ad-88a5-fccfc794f4ea response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150972025-03-04T03:31:46.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a43df60-0547-42c1-b8fd-1a9e1565143c roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150982025-03-04T03:31:46.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a43df60-0547-42c1-b8fd-1a9e1565143c resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
150992025-03-04T03:31:46.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2153 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 8a43df60-0547-42c1-b8fd-1a9e1565143c response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
151002025-03-04T03:31:47.207ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
151012025-03-04T03:31:47.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
151022025-03-04T03:31:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
151032025-03-04T03:31:47.208ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
151042025-03-04T03:31:47.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
151052025-03-04T03:31:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
151062025-03-04T03:31:47.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
151072025-03-04T03:31:47.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151082025-03-04T03:31:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
151092025-03-04T03:31:47.211ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.767336ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
151102025-03-04T03:31:47.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
151112025-03-04T03:31:47.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
151122025-03-04T03:31:47.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
151132025-03-04T03:31:47.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
151142025-03-04T03:31:47.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
151152025-03-04T03:31:47.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151162025-03-04T03:31:47.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1763ad9e-9534-4579-89c0-6dd7e78998ce, lookup_type: ById(1763ad9e-9534-4579-89c0-6dd7e78998ce) } result = Ok(())
151172025-03-04T03:31:47.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
151182025-03-04T03:31:47.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
151192025-03-04T03:31:47.229ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce original_time_created = 2025-03-04 03:17:57.947579 UTC target_id = 1763ad9e-9534-4579-89c0-6dd7e78998ce time_created = 2025-03-04 03:17:57.947579 UTC
151202025-03-04T03:31:47.229ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.554105ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1763ad9e-9534-4579-89c0-6dd7e78998ce","time_created":"2025-03-04T03:17:57.947579Z"}
151212025-03-04T03:31:47.477ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b1216e4-59d6-4607-bd50-0d6d54638806 roles = RoleSet { roles: {} } uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
151222025-03-04T03:31:47.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b1216e4-59d6-4607-bd50-0d6d54638806 resource = Database result = Ok(()) uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
151232025-03-04T03:31:47.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1860 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:63537 req_id = 7b1216e4-59d6-4607-bd50-0d6d54638806 response_code = 200 uri = /probes/5525e426-7299-43ab-ad2b-346968b24e1c
151242025-03-04T03:31:47.893ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:42147
151252025-03-04T03:31:47.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42147 req_id = 2dc859db-aa4b-4b72-acd9-3274373fedcb roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
151262025-03-04T03:31:47.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42147 req_id = 2dc859db-aa4b-4b72-acd9-3274373fedcb resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
151272025-03-04T03:31:47.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3616 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42147 req_id = 2dc859db-aa4b-4b72-acd9-3274373fedcb response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?sort_by=id_ascending
151282025-03-04T03:31:47.897ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42147 req_id = 064c0dbc-ea27-4b10-ad6f-4437c6389bec roles = RoleSet { roles: {} } uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
151292025-03-04T03:31:47.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42147 req_id = 064c0dbc-ea27-4b10-ad6f-4437c6389bec resource = Database result = Ok(()) uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D
151302025-03-04T03:31:47.898ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 511 local_addr = [fd00:1122:3344:101::c]:40647 method = GET remote_addr = [fd00:1122:3344:101::d]:64466 req_id = 10fde182-f314-476f-a2d5-096c1d54bcb7 response_code = 200 uri = /c28b34ab-1b39-4325-87ad-c4ccf5b009d5
151312025-03-04T03:31:47.898ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1789 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42147 req_id = 064c0dbc-ea27-4b10-ad6f-4437c6389bec response_code = 200 uri = /metrics/collectors/ea0fb0f5-7b4c-4550-9130-b713c9ef0c9a/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkODQ4NDQ1Mi03ZDQzLTQyYWMtYTU4OC01NzViNmQzMWI4ODIifX0%3D