Max level shown:
1 [ Nov 14 04:45:59 Enabled. ]
2 [ Nov 14 04:45:59 Rereading configuration. ]
3 [ Nov 14 04:46:01 Rereading configuration. ]
4 [ Nov 14 04:46:10 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/omicron-nexus/bin/nexus /var/svc/manifest/site/nexus/config.toml &"). ]
5 [ Nov 14 04:46:10 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72024-11-14T04:46:10.727ZDEBGnexus: registered DTrace probes
82024-11-14T04:46:10.729ZINFOnexus: setting up nexus server file = nexus/src/lib.rs:74
92024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Action file = nexus/auth/src/authz/oso_generic.rs:68
102024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = AnyActor file = nexus/auth/src/authz/oso_generic.rs:68
112024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = AuthenticatedActor file = nexus/auth/src/authz/oso_generic.rs:68
122024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = BlueprintConfig file = nexus/auth/src/authz/oso_generic.rs:68
132024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Database file = nexus/auth/src/authz/oso_generic.rs:68
142024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = DnsConfig file = nexus/auth/src/authz/oso_generic.rs:68
152024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Fleet file = nexus/auth/src/authz/oso_generic.rs:68
162024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Inventory file = nexus/auth/src/authz/oso_generic.rs:68
172024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = IpPoolList file = nexus/auth/src/authz/oso_generic.rs:68
182024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = ConsoleSessionList file = nexus/auth/src/authz/oso_generic.rs:68
192024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequestList file = nexus/auth/src/authz/oso_generic.rs:68
202024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = SiloCertificateList file = nexus/auth/src/authz/oso_generic.rs:68
212024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = SiloIdentityProviderList file = nexus/auth/src/authz/oso_generic.rs:68
222024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = SiloUserList file = nexus/auth/src/authz/oso_generic.rs:68
232024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Project file = nexus/auth/src/authz/oso_generic.rs:68
242024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Disk file = nexus/auth/src/authz/oso_generic.rs:68
252024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Snapshot file = nexus/auth/src/authz/oso_generic.rs:68
262024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = ProjectImage file = nexus/auth/src/authz/oso_generic.rs:68
272024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Instance file = nexus/auth/src/authz/oso_generic.rs:68
282024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = IpPool file = nexus/auth/src/authz/oso_generic.rs:68
292024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = InstanceNetworkInterface file = nexus/auth/src/authz/oso_generic.rs:68
302024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = Vpc file = nexus/auth/src/authz/oso_generic.rs:68
312024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = VpcRouter file = nexus/auth/src/authz/oso_generic.rs:68
322024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = InternetGateway file = nexus/auth/src/authz/oso_generic.rs:68
332024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpPool file = nexus/auth/src/authz/oso_generic.rs:68
342024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpAddress file = nexus/auth/src/authz/oso_generic.rs:68
352024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = RouterRoute file = nexus/auth/src/authz/oso_generic.rs:68
362024-11-14T04:46:10.739ZINFOnexus (ServerContext): registering Oso class class = VpcSubnet file = nexus/auth/src/authz/oso_generic.rs:68
372024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = FloatingIp file = nexus/auth/src/authz/oso_generic.rs:68
382024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = Image file = nexus/auth/src/authz/oso_generic.rs:68
392024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = SiloImage file = nexus/auth/src/authz/oso_generic.rs:68
402024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = AddressLot file = nexus/auth/src/authz/oso_generic.rs:68
412024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = Blueprint file = nexus/auth/src/authz/oso_generic.rs:68
422024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = LoopbackAddress file = nexus/auth/src/authz/oso_generic.rs:68
432024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = Certificate file = nexus/auth/src/authz/oso_generic.rs:68
442024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = ConsoleSession file = nexus/auth/src/authz/oso_generic.rs:68
452024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequest file = nexus/auth/src/authz/oso_generic.rs:68
462024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = DeviceAccessToken file = nexus/auth/src/authz/oso_generic.rs:68
472024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = PhysicalDisk file = nexus/auth/src/authz/oso_generic.rs:68
482024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = Rack file = nexus/auth/src/authz/oso_generic.rs:68
492024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = RoleBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
502024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = SshKey file = nexus/auth/src/authz/oso_generic.rs:68
512024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = Silo file = nexus/auth/src/authz/oso_generic.rs:68
522024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = SiloUser file = nexus/auth/src/authz/oso_generic.rs:68
532024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = SiloGroup file = nexus/auth/src/authz/oso_generic.rs:68
542024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = IdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
552024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = SamlIdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
562024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = Sled file = nexus/auth/src/authz/oso_generic.rs:68
572024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = TufRepo file = nexus/auth/src/authz/oso_generic.rs:68
582024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = TufArtifact file = nexus/auth/src/authz/oso_generic.rs:68
592024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = Zpool file = nexus/auth/src/authz/oso_generic.rs:68
602024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = Service file = nexus/auth/src/authz/oso_generic.rs:68
612024-11-14T04:46:10.740ZINFOnexus (ServerContext): registering Oso class class = UserBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
622024-11-14T04:46:10.740ZINFOnexus (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n file = nexus/auth/src/authz/oso_generic.rs:89
632024-11-14T04:46:10.756ZINFOnexus (ServerContext): Setting up resolver using DNS servers for subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344::, width: 48 } } file = nexus/src/context.rs:219
642024-11-14T04:46:10.759ZINFOnexus (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
652024-11-14T04:46:10.762ZINFOnexus (ServerContext): Setting up qorb database pool from DNS dns_addrs = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = nexus/src/context.rs:275
662024-11-14T04:46:10.771ZDEBGnexus (ServerContext): registered USDT probes
672024-11-14T04:46:10.899ZINFOnexus: Database schema version is up to date desired_version = 114.0.0 file = nexus/db-queries/src/db/datastore/db_metadata.rs:145 found_version = 114.0.0
682024-11-14T04:46:10.902ZINFOnexus (SEC): SEC running file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:813 sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
692024-11-14T04:46:10.902ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 949, tv_nsec: 674658431 } })
702024-11-14T04:46:10.902ZINFOnexus (Nexus): Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1131
712024-11-14T04:46:10.953ZINFOnexus (Nexus): determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
722024-11-14T04:46:10.954ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
732024-11-14T04:46:10.958ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d77e9d3-8130-4765-8c26-a8b42f629bda", "content-length": "26", "date": "Thu, 14 Nov 2024 04:46:10 GMT"} })
742024-11-14T04:46:10.958ZINFOnexus (Nexus): identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
752024-11-14T04:46:10.959ZINFOnexus (Nexus): completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
762024-11-14T04:46:10.992ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 949, tv_nsec: 764231935 } })
772024-11-14T04:46:10.992ZINFOnexus (Nexus): Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1131
782024-11-14T04:46:11.024ZINFOnexus (Nexus): determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
792024-11-14T04:46:11.024ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
802024-11-14T04:46:11.024ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a07c399-eeb0-4add-8ce1-dc5f84ab1b08", "content-length": "26", "date": "Thu, 14 Nov 2024 04:46:10 GMT"} })
812024-11-14T04:46:11.024ZINFOnexus (Nexus): identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
822024-11-14T04:46:11.024ZINFOnexus (Nexus): completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
832024-11-14T04:46:11.093ZDEBGnexus (clickhouse-client): registered USDT probes id = 35bdc7df-3933-4bd5-92c2-27640f61cb65
842024-11-14T04:46:11.093ZDEBGnexus (clickhouse-client): registered USDT probes id = 35bdc7df-3933-4bd5-92c2-27640f61cb65
852024-11-14T04:46:11.104ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
862024-11-14T04:46:11.120ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
872024-11-14T04:46:11.122ZDEBGnexus (DataLoader): attempting to create built-in users
882024-11-14T04:46:11.122ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
892024-11-14T04:46:11.123ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
902024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /artifacts/{kind}/{name}/{version} versions = all
912024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /bgtasks versions = all
922024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /bgtasks/activate versions = all
932024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /bgtasks/view/{bgtask_name} versions = all
942024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /clickhouse/policy versions = all
952024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /clickhouse/policy versions = all
962024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
972024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
982024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
992024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1002024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1012024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /demo-saga versions = all
1022024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1032024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /deployment/blueprints/all versions = all
1042024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1052024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1062024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /deployment/blueprints/import versions = all
1072024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /deployment/blueprints/regenerate versions = all
1082024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /deployment/blueprints/target versions = all
1092024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /deployment/blueprints/target versions = all
1102024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /deployment/blueprints/target/enabled versions = all
1112024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1122024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /disks/{disk_id} versions = all
1132024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /instances/{instance_id}/migrate versions = all
1142024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /metrics/collectors versions = all
1152024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1162024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /metrics/producers versions = all
1172024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1182024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /physical-disk/expunge versions = all
1192024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /probes/{sled} versions = all
1202024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1212024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /sagas versions = all
1222024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /sagas/{saga_id} versions = all
1232024-11-14T04:46:11.144ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /sled-agents/{sled_id} versions = all
1242024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /sled-agents/{sled_id} versions = all
1252024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1262024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /sleds/add versions = all
1272024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /sleds/expunge versions = all
1282024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /sleds/uninitialized versions = all
1292024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /switch/{switch_id} versions = all
1302024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /vmms/{propolis_id} versions = all
1312024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1322024-11-14T04:46:11.145ZINFOnexus (dropshot_internal): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::b]:12221
1332024-11-14T04:46:11.145ZDEBGnexus (dropshot_internal): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::b]:12221
1342024-11-14T04:46:11.145ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1352024-11-14T04:46:11.145ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1362024-11-14T04:46:11.192ZINFOnexus (DataLoader): created 0 built-in users file = nexus/db-queries/src/db/datastore/silo_user.rs:394
1372024-11-14T04:46:11.192ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1382024-11-14T04:46:11.194ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1392024-11-14T04:46:11.194ZDEBGnexus (DataLoader): attempting to create built-in roles
1402024-11-14T04:46:11.194ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1412024-11-14T04:46:11.194ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1422024-11-14T04:46:11.228ZINFOnexus (DataLoader): created 0 built-in roles file = nexus/db-queries/src/db/datastore/role.rs:85
1432024-11-14T04:46:11.228ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1442024-11-14T04:46:11.229ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1452024-11-14T04:46:11.229ZDEBGnexus (DataLoader): attempting to create built-in role assignments
1462024-11-14T04:46:11.229ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1472024-11-14T04:46:11.229ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1482024-11-14T04:46:11.529ZWARNnexus: Cannot look up rack: Object (of type ById(f971eb12-0fea-4084-96d3-74198bddd289)) not found: rack file = nexus/src/app/rack.rs:774
1492024-11-14T04:46:11.638ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:34258
1502024-11-14T04:46:11.659ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1512024-11-14T04:46:11.659ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1522024-11-14T04:46:11.710ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1532024-11-14T04:46:11.712ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1542024-11-14T04:46:11.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1552024-11-14T04:46:11.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1562024-11-14T04:46:11.832ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1572024-11-14T04:46:11.832ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1582024-11-14T04:46:11.932ZINFOnexus (dropshot_internal): set cockroachdb setting actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:124 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be setting = cluster.preserve_downgrade_option uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete value = 22.1
1592024-11-14T04:46:11.932ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1602024-11-14T04:46:11.932ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1612024-11-14T04:46:11.953ZINFOnexus (dropshot_internal): request completed error_message_external = not found: rack with id "f971eb12-0fea-4084-96d3-74198bddd289" error_message_internal = not found: rack with id "f971eb12-0fea-4084-96d3-74198bddd289" file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 314454 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 96030d57-c095-45df-b920-8840859d23be response_code = 404 uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1622024-11-14T04:46:12.804ZINFOnexus (DataLoader): created 0 built-in role assignments file = nexus/db-queries/src/db/datastore/role.rs:114
1632024-11-14T04:46:12.804ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1642024-11-14T04:46:12.805ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1652024-11-14T04:46:12.805ZDEBGnexus (DataLoader): attempting to create built-in silos
1662024-11-14T04:46:12.805ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1672024-11-14T04:46:12.805ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1682024-11-14T04:46:12.864ZINFOnexus (DataLoader): created 0 built-in silos file = nexus/db-queries/src/db/datastore/silo.rs:95
1692024-11-14T04:46:12.864ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1702024-11-14T04:46:12.864ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1712024-11-14T04:46:12.893ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1722024-11-14T04:46:12.894ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1732024-11-14T04:46:12.894ZDEBGnexus (DataLoader): attempting to create built-in projects
1742024-11-14T04:46:12.894ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1752024-11-14T04:46:12.894ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1762024-11-14T04:46:12.946ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1772024-11-14T04:46:12.946ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1782024-11-14T04:46:13.048ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1792024-11-14T04:46:13.048ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1802024-11-14T04:46:13.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1812024-11-14T04:46:13.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1822024-11-14T04:46:13.497ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1832024-11-14T04:46:13.498ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1842024-11-14T04:46:13.498ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1852024-11-14T04:46:13.498ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1862024-11-14T04:46:13.530ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1872024-11-14T04:46:13.530ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1882024-11-14T04:46:13.532ZWARNnexus: Cannot look up rack: Object (of type ById(f971eb12-0fea-4084-96d3-74198bddd289)) not found: rack file = nexus/src/app/rack.rs:774
1892024-11-14T04:46:13.548ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1902024-11-14T04:46:13.548ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1912024-11-14T04:46:13.551ZINFOnexus (dropshot_internal): request completed error_message_external = not found: rack with id "f971eb12-0fea-4084-96d3-74198bddd289" error_message_internal = not found: rack with id "f971eb12-0fea-4084-96d3-74198bddd289" file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 122987 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = f13209ed-a61c-44f5-9641-6431ecc43f3b response_code = 404 uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
1922024-11-14T04:46:13.784ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1932024-11-14T04:46:13.786ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1942024-11-14T04:46:13.786ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1952024-11-14T04:46:13.787ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1962024-11-14T04:46:13.829ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1972024-11-14T04:46:13.829ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1982024-11-14T04:46:13.831ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1992024-11-14T04:46:13.832ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2002024-11-14T04:46:13.832ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2012024-11-14T04:46:13.832ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2022024-11-14T04:46:13.902ZINFOnexus (DataLoader): created built-in services project file = nexus/db-queries/src/db/datastore/project.rs:122
2032024-11-14T04:46:13.902ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2042024-11-14T04:46:13.903ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2052024-11-14T04:46:13.903ZDEBGnexus (DataLoader): attempting to create built-in VPCs
2062024-11-14T04:46:13.903ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2072024-11-14T04:46:13.903ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2082024-11-14T04:46:13.917ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2092024-11-14T04:46:13.918ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2102024-11-14T04:46:13.951ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2112024-11-14T04:46:13.951ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2122024-11-14T04:46:13.959ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2132024-11-14T04:46:13.959ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2142024-11-14T04:46:13.960ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2152024-11-14T04:46:13.960ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2162024-11-14T04:46:13.961ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2172024-11-14T04:46:13.964ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2182024-11-14T04:46:13.964ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2192024-11-14T04:46:13.964ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2202024-11-14T04:46:13.966ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2212024-11-14T04:46:13.966ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2222024-11-14T04:46:13.967ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2232024-11-14T04:46:13.968ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2242024-11-14T04:46:13.968ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2252024-11-14T04:46:13.971ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2262024-11-14T04:46:13.971ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2272024-11-14T04:46:13.971ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2282024-11-14T04:46:14.049ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2292024-11-14T04:46:14.049ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2302024-11-14T04:46:14.051ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2312024-11-14T04:46:14.051ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2322024-11-14T04:46:14.053ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2332024-11-14T04:46:14.053ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2342024-11-14T04:46:14.054ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2352024-11-14T04:46:14.057ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2362024-11-14T04:46:14.057ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2372024-11-14T04:46:14.057ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2382024-11-14T04:46:14.088ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2392024-11-14T04:46:14.088ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2402024-11-14T04:46:14.122ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2412024-11-14T04:46:14.122ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2422024-11-14T04:46:14.146ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2432024-11-14T04:46:14.146ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2442024-11-14T04:46:14.148ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2452024-11-14T04:46:14.148ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2462024-11-14T04:46:14.149ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2472024-11-14T04:46:14.149ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2482024-11-14T04:46:14.150ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2492024-11-14T04:46:14.153ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2502024-11-14T04:46:14.153ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2512024-11-14T04:46:14.153ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2522024-11-14T04:46:14.193ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2532024-11-14T04:46:14.193ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2542024-11-14T04:46:14.195ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2552024-11-14T04:46:14.195ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2562024-11-14T04:46:14.197ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2572024-11-14T04:46:14.197ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2582024-11-14T04:46:14.198ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2592024-11-14T04:46:14.202ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2602024-11-14T04:46:14.202ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2612024-11-14T04:46:14.202ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2622024-11-14T04:46:14.272ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2632024-11-14T04:46:14.273ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2642024-11-14T04:46:14.274ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2652024-11-14T04:46:14.274ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2662024-11-14T04:46:14.275ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2672024-11-14T04:46:14.276ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2682024-11-14T04:46:14.277ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2692024-11-14T04:46:14.280ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2702024-11-14T04:46:14.281ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2712024-11-14T04:46:14.281ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2722024-11-14T04:46:14.402ZDEBGnexus (DataLoader): attempting to create built-in VPC firewall rules
2732024-11-14T04:46:14.402ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2742024-11-14T04:46:14.403ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2752024-11-14T04:46:14.417ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2762024-11-14T04:46:14.418ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772024-11-14T04:46:14.425ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2782024-11-14T04:46:14.425ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2792024-11-14T04:46:14.427ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2802024-11-14T04:46:14.427ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2812024-11-14T04:46:14.428ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2822024-11-14T04:46:14.428ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2832024-11-14T04:46:14.429ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2842024-11-14T04:46:14.429ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2852024-11-14T04:46:14.430ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2862024-11-14T04:46:14.433ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2872024-11-14T04:46:14.433ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2882024-11-14T04:46:14.433ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2892024-11-14T04:46:14.434ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2902024-11-14T04:46:14.435ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2912024-11-14T04:46:14.436ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2922024-11-14T04:46:14.436ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2932024-11-14T04:46:14.437ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2942024-11-14T04:46:14.440ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2952024-11-14T04:46:14.440ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2962024-11-14T04:46:14.440ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2972024-11-14T04:46:14.469ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2982024-11-14T04:46:14.469ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2992024-11-14T04:46:14.470ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3002024-11-14T04:46:14.471ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3012024-11-14T04:46:14.472ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3022024-11-14T04:46:14.472ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3032024-11-14T04:46:14.473ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3042024-11-14T04:46:14.478ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3052024-11-14T04:46:14.478ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3062024-11-14T04:46:14.478ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3072024-11-14T04:46:14.700ZINFOnexus (DataLoader): created built-in VPC firewall rules file = nexus/db-queries/src/db/datastore/vpc.rs:282
3082024-11-14T04:46:14.701ZDEBGnexus (DataLoader): attempting to create built-in VPC Subnets
3092024-11-14T04:46:14.701ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3102024-11-14T04:46:14.701ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3112024-11-14T04:46:14.720ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3122024-11-14T04:46:14.720ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3132024-11-14T04:46:14.722ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3142024-11-14T04:46:14.722ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152024-11-14T04:46:14.724ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3162024-11-14T04:46:14.725ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3172024-11-14T04:46:14.726ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3182024-11-14T04:46:14.726ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192024-11-14T04:46:14.728ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3202024-11-14T04:46:14.728ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3212024-11-14T04:46:14.729ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3222024-11-14T04:46:14.732ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3232024-11-14T04:46:14.732ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3242024-11-14T04:46:14.732ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3252024-11-14T04:46:14.764ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3262024-11-14T04:46:14.764ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3272024-11-14T04:46:14.765ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3282024-11-14T04:46:14.766ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3292024-11-14T04:46:14.767ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3302024-11-14T04:46:14.767ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3312024-11-14T04:46:14.769ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3322024-11-14T04:46:14.771ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3332024-11-14T04:46:14.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3342024-11-14T04:46:14.853ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3352024-11-14T04:46:14.853ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3362024-11-14T04:46:14.853ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372024-11-14T04:46:14.854ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3382024-11-14T04:46:14.855ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3392024-11-14T04:46:14.855ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3402024-11-14T04:46:14.855ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3412024-11-14T04:46:14.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3422024-11-14T04:46:14.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3432024-11-14T04:46:14.898ZINFOnexus (dropshot_internal): request completed error_message_external = not found: rack with id "f971eb12-0fea-4084-96d3-74198bddd289" error_message_internal = not found: rack with id "f971eb12-0fea-4084-96d3-74198bddd289" file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 46389 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 8053157a-6876-46be-b4f7-7eb4e43adb3b response_code = 404 uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
3442024-11-14T04:46:14.957ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3452024-11-14T04:46:14.957ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3462024-11-14T04:46:14.974ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3472024-11-14T04:46:14.974ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3482024-11-14T04:46:14.975ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3492024-11-14T04:46:14.976ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3502024-11-14T04:46:14.977ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3512024-11-14T04:46:14.977ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3522024-11-14T04:46:14.979ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3532024-11-14T04:46:14.982ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3542024-11-14T04:46:14.982ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3552024-11-14T04:46:14.982ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3562024-11-14T04:46:15.115ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3572024-11-14T04:46:15.115ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3582024-11-14T04:46:15.117ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3592024-11-14T04:46:15.117ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3602024-11-14T04:46:15.118ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3612024-11-14T04:46:15.118ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3622024-11-14T04:46:15.120ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3632024-11-14T04:46:15.120ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3642024-11-14T04:46:15.121ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3652024-11-14T04:46:15.121ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3662024-11-14T04:46:15.123ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3672024-11-14T04:46:15.123ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3682024-11-14T04:46:15.124ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3692024-11-14T04:46:15.125ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3702024-11-14T04:46:15.126ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3712024-11-14T04:46:15.129ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ById(001de000-c470-4000-8000-000000000002) } result = Ok(())
3722024-11-14T04:46:15.129ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3732024-11-14T04:46:15.129ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3742024-11-14T04:46:15.140ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3752024-11-14T04:46:15.140ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3762024-11-14T04:46:15.142ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3772024-11-14T04:46:15.142ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3782024-11-14T04:46:15.144ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3792024-11-14T04:46:15.144ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3802024-11-14T04:46:15.146ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3812024-11-14T04:46:15.146ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3822024-11-14T04:46:15.147ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3832024-11-14T04:46:15.147ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3842024-11-14T04:46:15.148ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3852024-11-14T04:46:15.148ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3862024-11-14T04:46:15.149ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3872024-11-14T04:46:15.153ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ById(001de000-c470-4000-8000-000000000003) } result = Ok(())
3882024-11-14T04:46:15.153ZINFOnexus (DataLoader): created built-in services vpc subnets file = nexus/db-queries/src/db/datastore/vpc.rs:346
3892024-11-14T04:46:15.153ZINFOnexus (DataLoader): created built-in services vpc file = nexus/db-queries/src/db/datastore/vpc.rs:224
3902024-11-14T04:46:15.153ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3912024-11-14T04:46:15.154ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3922024-11-14T04:46:15.154ZDEBGnexus (DataLoader): attempting to create silo users
3932024-11-14T04:46:15.154ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3942024-11-14T04:46:15.154ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952024-11-14T04:46:15.171ZINFOnexus (DataLoader): created 2 silo users file = nexus/db-queries/src/db/datastore/silo_user.rs:419
3962024-11-14T04:46:15.171ZDEBGnexus (DataLoader): attempting to create silo user role assignments
3972024-11-14T04:46:15.171ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3982024-11-14T04:46:15.171ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992024-11-14T04:46:15.182ZINFOnexus (DataLoader): created 2 silo user role assignments file = nexus/db-queries/src/db/datastore/silo_user.rs:446
4002024-11-14T04:46:15.182ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4012024-11-14T04:46:15.182ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4022024-11-14T04:46:15.187ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4032024-11-14T04:46:15.187ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4042024-11-14T04:46:15.202ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4052024-11-14T04:46:15.203ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4062024-11-14T04:46:15.204ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4072024-11-14T04:46:15.205ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
4082024-11-14T04:46:15.205ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4092024-11-14T04:46:15.205ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4102024-11-14T04:46:15.227ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4112024-11-14T04:46:15.228ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4122024-11-14T04:46:15.229ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4132024-11-14T04:46:15.230ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
4142024-11-14T04:46:15.230ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4152024-11-14T04:46:15.230ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4162024-11-14T04:46:15.299ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4172024-11-14T04:46:15.299ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4182024-11-14T04:46:15.315ZINFOnexus: populate complete file = nexus/src/app/mod.rs:555
4192024-11-14T04:46:15.315ZINFOnexus: activating background tasks file = nexus/src/app/mod.rs:564
4202024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
4212024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
4222024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
4232024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
4242024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
4252024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
4262024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
4272024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
4282024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
4292024-11-14T04:46:15.320ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:74 reason = no blueprint
4302024-11-14T04:46:15.320ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector file = nexus/src/app/background/tasks/crdb_node_id_collector.rs:78 reason = no blueprint
4312024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
4322024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
4332024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
4342024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4352024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4362024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
4372024-11-14T04:46:15.320ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
4382024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4392024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
4402024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4412024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
4422024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
4432024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4442024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
4452024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4462024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
4472024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4482024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
4492024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
4502024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
4512024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4522024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
4532024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4542024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
4552024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
4562024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4572024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4582024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
4592024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
4602024-11-14T04:46:15.320ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
4612024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4622024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4632024-11-14T04:46:15.320ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4642024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
4652024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
4662024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
4672024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
4682024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
4692024-11-14T04:46:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
4702024-11-14T04:46:15.321ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
4712024-11-14T04:46:15.321ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
4722024-11-14T04:46:15.321ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 124.284\u{b5}s iteration = 1 status = {"error":"no blueprint"}
4732024-11-14T04:46:15.321ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 69.722\u{b5}s iteration = 1 status = {"error":"no blueprint"}
4742024-11-14T04:46:15.321ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
4752024-11-14T04:46:15.321ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4762024-11-14T04:46:15.321ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 987.259\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
4772024-11-14T04:46:15.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4782024-11-14T04:46:15.321ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:36:15.321507071 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4792024-11-14T04:46:15.321ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4802024-11-14T04:46:15.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4812024-11-14T04:46:15.322ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4822024-11-14T04:46:15.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4832024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4842024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4852024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
4862024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4872024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4882024-11-14T04:46:15.323ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
4892024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4902024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4912024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4922024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4932024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4942024-11-14T04:46:15.323ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
4952024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 2.795111ms iteration = 1 status = {"error":"no config nor servers"}
4962024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4972024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
4982024-11-14T04:46:15.323ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 954, tv_nsec: 95835612 } })
4992024-11-14T04:46:15.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
5002024-11-14T04:46:15.323ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 954, tv_nsec: 96023687 } })
5012024-11-14T04:46:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5022024-11-14T04:46:15.324ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5032024-11-14T04:46:15.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5042024-11-14T04:46:15.325ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5052024-11-14T04:46:15.326ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5062024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5072024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5082024-11-14T04:46:15.328ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external dns_group = external file = nexus/src/app/background/tasks/dns_servers.rs:113
5092024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.149265ms iteration = 1 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
5102024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
5112024-11-14T04:46:15.328ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
5122024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 4.54\u{b5}s iteration = 2 status = {"error":"no config"}
5132024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
5142024-11-14T04:46:15.328ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal dns_group = internal file = nexus/src/app/background/tasks/dns_servers.rs:113
5152024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
5162024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5172024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.058903ms iteration = 1 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
5182024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
5192024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5202024-11-14T04:46:15.328ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
5212024-11-14T04:46:15.328ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 7.02\u{b5}s iteration = 2 status = {"error":"no config"}
5222024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
5232024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5242024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5252024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5262024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5272024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5282024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5292024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5302024-11-14T04:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5312024-11-14T04:46:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5322024-11-14T04:46:15.330ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
5332024-11-14T04:46:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5342024-11-14T04:46:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5352024-11-14T04:46:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5362024-11-14T04:46:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
5372024-11-14T04:46:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5382024-11-14T04:46:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5392024-11-14T04:46:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5402024-11-14T04:46:15.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5412024-11-14T04:46:15.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5422024-11-14T04:46:15.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5432024-11-14T04:46:15.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5442024-11-14T04:46:15.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5452024-11-14T04:46:15.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5462024-11-14T04:46:15.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5472024-11-14T04:46:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5482024-11-14T04:46:15.333ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5492024-11-14T04:46:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5502024-11-14T04:46:15.376ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5512024-11-14T04:46:15.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5522024-11-14T04:46:15.391ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 954, tv_nsec: 163539753 } })
5532024-11-14T04:46:15.391ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
5542024-11-14T04:46:15.415ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5552024-11-14T04:46:15.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5562024-11-14T04:46:15.431ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5572024-11-14T04:46:15.432ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5582024-11-14T04:46:15.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5592024-11-14T04:46:15.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5602024-11-14T04:46:15.434ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5612024-11-14T04:46:15.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5622024-11-14T04:46:15.436ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 115.116414ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
5632024-11-14T04:46:15.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5642024-11-14T04:46:15.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5652024-11-14T04:46:15.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5662024-11-14T04:46:15.438ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
5672024-11-14T04:46:15.438ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5682024-11-14T04:46:15.438ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "455a6169-847c-47d0-a1a4-fd8587bc2c6a", "content-length": "26", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
5692024-11-14T04:46:15.438ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5702024-11-14T04:46:15.440ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5712024-11-14T04:46:15.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5722024-11-14T04:46:15.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5732024-11-14T04:46:15.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5742024-11-14T04:46:15.452ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5752024-11-14T04:46:15.452ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:36:15.321507071 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5762024-11-14T04:46:15.452ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 132.300292ms iteration = 1 status = {"expiration":"2024-11-14T04:36:15.321507071Z","pruned":[]}
5772024-11-14T04:46:15.455ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5782024-11-14T04:46:15.455ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5792024-11-14T04:46:15.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5802024-11-14T04:46:15.475ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5812024-11-14T04:46:15.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5822024-11-14T04:46:15.477ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5832024-11-14T04:46:15.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5842024-11-14T04:46:15.483ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5852024-11-14T04:46:15.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
5862024-11-14T04:46:15.487ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5872024-11-14T04:46:15.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5882024-11-14T04:46:15.487ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 166.851482ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5892024-11-14T04:46:15.487ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 167.027097ms iteration = 1 status = {}
5902024-11-14T04:46:15.495ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5912024-11-14T04:46:15.496ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5922024-11-14T04:46:15.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5932024-11-14T04:46:15.497ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5942024-11-14T04:46:15.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5952024-11-14T04:46:15.500ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5962024-11-14T04:46:15.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5972024-11-14T04:46:15.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5982024-11-14T04:46:15.502ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
5992024-11-14T04:46:15.503ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 954, tv_nsec: 275629679 } })
6002024-11-14T04:46:15.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6012024-11-14T04:46:15.506ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:79 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
6022024-11-14T04:46:15.506ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 186.033597ms iteration = 1 status = {"error":"rack not yet initialized: f971eb12-0fea-4084-96d3-74198bddd289"}
6032024-11-14T04:46:15.508ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6042024-11-14T04:46:15.508ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6052024-11-14T04:46:15.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6062024-11-14T04:46:15.512ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 954, tv_nsec: 284717352 } })
6072024-11-14T04:46:15.512ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6082024-11-14T04:46:15.513ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
6092024-11-14T04:46:15.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6102024-11-14T04:46:15.514ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 193.971167ms iteration = 1 status = {}
6112024-11-14T04:46:15.516ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6122024-11-14T04:46:15.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6132024-11-14T04:46:15.527ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6142024-11-14T04:46:15.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6152024-11-14T04:46:15.529ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6162024-11-14T04:46:15.530ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6172024-11-14T04:46:15.532ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6182024-11-14T04:46:15.533ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6192024-11-14T04:46:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6202024-11-14T04:46:15.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6212024-11-14T04:46:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6222024-11-14T04:46:15.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6232024-11-14T04:46:15.535ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6242024-11-14T04:46:15.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6252024-11-14T04:46:15.538ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6262024-11-14T04:46:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6272024-11-14T04:46:15.538ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 217.677734ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
6282024-11-14T04:46:15.539ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
6292024-11-14T04:46:15.539ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 218.980421ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
6302024-11-14T04:46:15.543ZWARNnexus (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker file = nexus/src/app/background/tasks/sync_service_zone_nat.rs:95
6312024-11-14T04:46:15.543ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 222.619616ms iteration = 1 status = {"error":"inventory collection is None"}
6322024-11-14T04:46:15.546ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6332024-11-14T04:46:15.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6342024-11-14T04:46:15.550ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6352024-11-14T04:46:15.550ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6362024-11-14T04:46:15.552ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6372024-11-14T04:46:15.552ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6382024-11-14T04:46:15.552ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6392024-11-14T04:46:15.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6402024-11-14T04:46:15.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 242.000648ms iteration = 1 status = {"errors":[],"records_set_to_done":[]}
6412024-11-14T04:46:15.566ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6422024-11-14T04:46:15.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6432024-11-14T04:46:15.569ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
6442024-11-14T04:46:15.569ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6452024-11-14T04:46:15.570ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e3ca7e8-3a4d-46a6-8831-281b69791274", "content-length": "26", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6462024-11-14T04:46:15.570ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6472024-11-14T04:46:15.572ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6482024-11-14T04:46:15.576ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
6492024-11-14T04:46:15.576ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
6502024-11-14T04:46:15.576ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
6512024-11-14T04:46:15.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf582bb-f14e-4f93-bec9-826b6704fc39", "content-length": "476", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6522024-11-14T04:46:15.578ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6532024-11-14T04:46:15.578ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
6542024-11-14T04:46:15.578ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
6552024-11-14T04:46:15.578ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 258.238938ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
6562024-11-14T04:46:15.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6572024-11-14T04:46:15.579ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed28e7ce-fec4-425e-807d-7f39f58979ac", "content-length": "734", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6582024-11-14T04:46:15.579ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
6592024-11-14T04:46:15.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fbbe3a8-4e34-4259-a80b-5973e5b02066", "content-length": "111", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6602024-11-14T04:46:15.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
6612024-11-14T04:46:15.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "558500f1-67ea-4731-854f-ac2c0e5ce82f", "content-length": "111", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6622024-11-14T04:46:15.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
6632024-11-14T04:46:15.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e99297c-ea71-457a-ab47-95c889e9db6f", "content-length": "167", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6642024-11-14T04:46:15.582ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
6652024-11-14T04:46:15.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a83626cb-2cdf-4e15-96f2-6a13cdd9fec4", "content-length": "167", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6662024-11-14T04:46:15.582ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
6672024-11-14T04:46:15.583ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
6682024-11-14T04:46:15.583ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 262.514852ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
6692024-11-14T04:46:15.583ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d654796e-b737-440a-8861-91d08c01a994", "content-length": "175", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6702024-11-14T04:46:15.583ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
6712024-11-14T04:46:15.584ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80322281-4195-4edb-b2e3-8e9710d56a7a", "content-length": "175", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6722024-11-14T04:46:15.584ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
6732024-11-14T04:46:15.586ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "188f8757-c0de-49ad-8a3d-9af0bb85b2e6", "content-length": "702", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6742024-11-14T04:46:15.586ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
6752024-11-14T04:46:15.587ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6762024-11-14T04:46:15.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6772024-11-14T04:46:15.598ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee3d971d-95c3-450b-9f49-aaf8e6ec56e8", "content-length": "727", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6782024-11-14T04:46:15.598ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
6792024-11-14T04:46:15.599ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5859361-89a3-4941-85c4-cc2c14fa35fd", "content-length": "729", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6802024-11-14T04:46:15.599ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
6812024-11-14T04:46:15.599ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6822024-11-14T04:46:15.599ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
6832024-11-14T04:46:15.599ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 278.511275ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
6842024-11-14T04:46:15.599ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33221904-e405-4255-8ebb-4a31ffb89a4d", "content-length": "728", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6852024-11-14T04:46:15.599ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
6862024-11-14T04:46:15.600ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "617e17ae-c610-4e29-802f-bf6d5d162cec", "content-length": "722", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6872024-11-14T04:46:15.600ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
6882024-11-14T04:46:15.600ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4643a586-e1d1-479d-8ac0-dfca6832bd31", "content-length": "109", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6892024-11-14T04:46:15.600ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
6902024-11-14T04:46:15.601ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc7f27c6-7270-47cf-bf5a-db4005fb714d", "content-length": "109", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6912024-11-14T04:46:15.601ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
6922024-11-14T04:46:15.601ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff9bbd74-8248-428e-aad8-18f3a559c15b", "content-length": "169", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6932024-11-14T04:46:15.601ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
6942024-11-14T04:46:15.601ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d37ce8da-463b-49f5-9a55-ee3ad5dd56cc", "content-length": "169", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6952024-11-14T04:46:15.602ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
6962024-11-14T04:46:15.602ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e232aaa-0b37-45c1-b8bf-27c1d67232d2", "content-length": "178", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6972024-11-14T04:46:15.602ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
6982024-11-14T04:46:15.602ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e44fdfeb-f12b-445e-80c4-8df83fffdc3b", "content-length": "178", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
6992024-11-14T04:46:15.602ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
7002024-11-14T04:46:15.603ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d03bbef8-97c7-4af2-a9f7-248be1e622eb", "content-length": "702", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
7012024-11-14T04:46:15.603ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
7022024-11-14T04:46:15.603ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88680929-8c12-4801-bc52-6cf24a4c79aa", "content-length": "727", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
7032024-11-14T04:46:15.603ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
7042024-11-14T04:46:15.603ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06afb3b2-eeae-45ed-a1b5-4f6625fd0517", "content-length": "729", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
7052024-11-14T04:46:15.603ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
7062024-11-14T04:46:15.603ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a70aea2-0a7f-4f3a-b1f1-6c788d7c4cbd", "content-length": "728", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
7072024-11-14T04:46:15.603ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7082024-11-14T04:46:15.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7092024-11-14T04:46:15.606ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7102024-11-14T04:46:15.609ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
7112024-11-14T04:46:15.609ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7122024-11-14T04:46:15.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7132024-11-14T04:46:15.613ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7142024-11-14T04:46:15.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7152024-11-14T04:46:15.616ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7162024-11-14T04:46:15.618ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7172024-11-14T04:46:15.619ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "84338e66-92f7-4ab9-8ea3-83d1cdbc805e", "content-length": "1", "date": "Thu, 14 Nov 2024 04:46:14 GMT"} })
7182024-11-14T04:46:15.620ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7192024-11-14T04:46:15.621ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7202024-11-14T04:46:15.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7212024-11-14T04:46:15.622ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7222024-11-14T04:46:15.622ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7232024-11-14T04:46:15.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7242024-11-14T04:46:15.623ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: f971eb12-0fea-4084-96d3-74198bddd289, lookup_type: ById(f971eb12-0fea-4084-96d3-74198bddd289) } result = Ok(())
7252024-11-14T04:46:15.623ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: f971eb12-0fea-4084-96d3-74198bddd289, time_created: 2024-11-14T04:46:15.187444Z, time_modified: 2024-11-14T04:46:15.187444Z }, initialized: false, tuf_base_url: None, rack_subnet: None } file = nexus/src/app/rack.rs:768
7262024-11-14T04:46:15.625ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7272024-11-14T04:46:15.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7282024-11-14T04:46:15.626ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7292024-11-14T04:46:15.626ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7302024-11-14T04:46:15.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7312024-11-14T04:46:15.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7322024-11-14T04:46:15.627ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7332024-11-14T04:46:15.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7342024-11-14T04:46:15.628ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 307.862666ms iteration = 1 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
7352024-11-14T04:46:15.629ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7362024-11-14T04:46:15.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7372024-11-14T04:46:15.631ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7382024-11-14T04:46:15.631ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7392024-11-14T04:46:15.631ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7402024-11-14T04:46:15.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7412024-11-14T04:46:15.631ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 311.009517ms iteration = 1 status = null
7422024-11-14T04:46:15.632ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7432024-11-14T04:46:15.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7442024-11-14T04:46:15.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 323.183269ms iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
7452024-11-14T04:46:15.652ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone file = nexus/src/app/background/tasks/external_endpoints.rs:48
7462024-11-14T04:46:15.652ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 331.801208ms iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
7472024-11-14T04:46:15.666ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 345.633909ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7482024-11-14T04:46:15.669ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7492024-11-14T04:46:15.672ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
7502024-11-14T04:46:15.672ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 351.18503ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
7512024-11-14T04:46:15.672ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
7522024-11-14T04:46:15.672ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7532024-11-14T04:46:15.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7542024-11-14T04:46:15.675ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
7552024-11-14T04:46:15.675ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
7562024-11-14T04:46:15.675ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 355.167215ms iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7572024-11-14T04:46:15.685ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7582024-11-14T04:46:15.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7592024-11-14T04:46:15.694ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 373.787604ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7602024-11-14T04:46:15.723ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7612024-11-14T04:46:15.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7622024-11-14T04:46:15.726ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7632024-11-14T04:46:15.727ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
7642024-11-14T04:46:15.727ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 407.231333ms iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
7652024-11-14T04:46:15.730ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7662024-11-14T04:46:15.730ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7672024-11-14T04:46:15.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7682024-11-14T04:46:15.732ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7692024-11-14T04:46:15.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7702024-11-14T04:46:15.734ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7712024-11-14T04:46:15.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7722024-11-14T04:46:15.737ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7732024-11-14T04:46:15.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7742024-11-14T04:46:15.740ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7752024-11-14T04:46:15.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7762024-11-14T04:46:15.741ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7772024-11-14T04:46:15.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7782024-11-14T04:46:15.742ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7792024-11-14T04:46:15.742ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 421.767444ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7802024-11-14T04:46:15.742ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7812024-11-14T04:46:15.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7822024-11-14T04:46:15.744ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7832024-11-14T04:46:15.747ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
7842024-11-14T04:46:15.747ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7852024-11-14T04:46:15.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7862024-11-14T04:46:15.750ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7872024-11-14T04:46:15.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7882024-11-14T04:46:15.750ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7892024-11-14T04:46:15.750ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7902024-11-14T04:46:15.750ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 430.196548ms iteration = 1 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7912024-11-14T04:46:15.763ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7922024-11-14T04:46:15.763ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7932024-11-14T04:46:15.766ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "14448933-60f7-4f98-8718-396676f704ec", "content-length": "312", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
7942024-11-14T04:46:15.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7952024-11-14T04:46:15.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7962024-11-14T04:46:15.769ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7972024-11-14T04:46:15.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7982024-11-14T04:46:15.801ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7992024-11-14T04:46:15.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8002024-11-14T04:46:15.803ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8012024-11-14T04:46:15.807ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8022024-11-14T04:46:15.807ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8032024-11-14T04:46:15.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8042024-11-14T04:46:15.813ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8052024-11-14T04:46:15.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8062024-11-14T04:46:15.815ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8072024-11-14T04:46:15.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8082024-11-14T04:46:15.825ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
8092024-11-14T04:46:15.825ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
8102024-11-14T04:46:15.827ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8112024-11-14T04:46:15.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8122024-11-14T04:46:15.830ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8132024-11-14T04:46:15.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8142024-11-14T04:46:15.833ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8152024-11-14T04:46:15.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8162024-11-14T04:46:15.836ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8172024-11-14T04:46:15.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8182024-11-14T04:46:15.838ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8192024-11-14T04:46:15.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8202024-11-14T04:46:15.839ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8212024-11-14T04:46:15.843ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8222024-11-14T04:46:15.843ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8232024-11-14T04:46:15.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8242024-11-14T04:46:15.849ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8252024-11-14T04:46:15.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8262024-11-14T04:46:15.851ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8272024-11-14T04:46:15.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8282024-11-14T04:46:15.853ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8292024-11-14T04:46:15.857ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8302024-11-14T04:46:15.857ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8312024-11-14T04:46:15.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8322024-11-14T04:46:15.863ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
8332024-11-14T04:46:15.863ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8342024-11-14T04:46:15.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8352024-11-14T04:46:15.865ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8362024-11-14T04:46:15.867ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
8372024-11-14T04:46:15.867ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8382024-11-14T04:46:15.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8392024-11-14T04:46:15.875ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
8402024-11-14T04:46:15.875ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8412024-11-14T04:46:15.885ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8422024-11-14T04:46:15.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8432024-11-14T04:46:15.887ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8442024-11-14T04:46:15.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8452024-11-14T04:46:15.889ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8462024-11-14T04:46:15.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8472024-11-14T04:46:15.890ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8482024-11-14T04:46:15.891ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b526a2f1-d6a7-4c74-9e43-db1387a8759a", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
8492024-11-14T04:46:15.891ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8502024-11-14T04:46:15.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8512024-11-14T04:46:15.892ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8522024-11-14T04:46:15.892ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8532024-11-14T04:46:15.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8542024-11-14T04:46:15.894ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8552024-11-14T04:46:15.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8562024-11-14T04:46:15.897ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8572024-11-14T04:46:15.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8582024-11-14T04:46:15.901ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8592024-11-14T04:46:15.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8602024-11-14T04:46:15.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8612024-11-14T04:46:15.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8622024-11-14T04:46:15.905ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8632024-11-14T04:46:15.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8642024-11-14T04:46:15.905ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
8652024-11-14T04:46:15.905ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
8662024-11-14T04:46:15.905ZDEBGnexus (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
8672024-11-14T04:46:15.931ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8682024-11-14T04:46:15.932ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "00dcc441-7018-4174-932e-510f3e443948", "content-length": "691", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
8692024-11-14T04:46:15.932ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
8702024-11-14T04:46:15.932ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
8712024-11-14T04:46:15.932ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
8722024-11-14T04:46:15.933ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 612.830936ms iteration = 1 status = {}
8732024-11-14T04:46:15.939ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8742024-11-14T04:46:15.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8752024-11-14T04:46:15.941ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8762024-11-14T04:46:15.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8772024-11-14T04:46:15.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8782024-11-14T04:46:15.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8792024-11-14T04:46:15.944ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8802024-11-14T04:46:15.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8812024-11-14T04:46:15.946ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8822024-11-14T04:46:15.950ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
8832024-11-14T04:46:15.950ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8842024-11-14T04:46:15.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8852024-11-14T04:46:15.954ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8862024-11-14T04:46:15.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8872024-11-14T04:46:15.957ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8882024-11-14T04:46:15.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8892024-11-14T04:46:15.959ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8902024-11-14T04:46:15.963ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
8912024-11-14T04:46:15.963ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8922024-11-14T04:46:15.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8932024-11-14T04:46:15.966ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8942024-11-14T04:46:15.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8952024-11-14T04:46:15.968ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8962024-11-14T04:46:15.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8972024-11-14T04:46:15.970ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8982024-11-14T04:46:15.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8992024-11-14T04:46:15.976ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9002024-11-14T04:46:15.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9012024-11-14T04:46:15.978ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9022024-11-14T04:46:15.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9032024-11-14T04:46:15.979ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9042024-11-14T04:46:15.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9052024-11-14T04:46:15.981ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9062024-11-14T04:46:15.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9072024-11-14T04:46:15.983ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9082024-11-14T04:46:15.988ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
9092024-11-14T04:46:15.988ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9102024-11-14T04:46:15.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9112024-11-14T04:46:15.991ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9122024-11-14T04:46:15.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9132024-11-14T04:46:15.992ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9142024-11-14T04:46:15.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9152024-11-14T04:46:15.994ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9162024-11-14T04:46:15.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9172024-11-14T04:46:15.997ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9182024-11-14T04:46:15.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9192024-11-14T04:46:15.999ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9202024-11-14T04:46:15.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9212024-11-14T04:46:16.001ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9222024-11-14T04:46:16.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9232024-11-14T04:46:16.002ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9242024-11-14T04:46:16.008ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ByName("boundary-ntp") } result = Ok(())
9252024-11-14T04:46:16.008ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9262024-11-14T04:46:16.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9272024-11-14T04:46:16.009ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9282024-11-14T04:46:16.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9292024-11-14T04:46:16.011ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9302024-11-14T04:46:16.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9312024-11-14T04:46:16.013ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9322024-11-14T04:46:16.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9332024-11-14T04:46:16.016ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9342024-11-14T04:46:16.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9352024-11-14T04:46:16.018ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9362024-11-14T04:46:16.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9372024-11-14T04:46:16.019ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9382024-11-14T04:46:16.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9392024-11-14T04:46:16.020ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9402024-11-14T04:46:16.020ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9412024-11-14T04:46:16.021ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9422024-11-14T04:46:16.022ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9432024-11-14T04:46:16.023ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9442024-11-14T04:46:16.023ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9452024-11-14T04:46:16.023ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9462024-11-14T04:46:16.025ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
9472024-11-14T04:46:16.025ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9482024-11-14T04:46:16.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9492024-11-14T04:46:16.026ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9502024-11-14T04:46:16.026ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9512024-11-14T04:46:16.027ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9522024-11-14T04:46:16.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9532024-11-14T04:46:16.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9542024-11-14T04:46:16.028ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9552024-11-14T04:46:16.028ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9562024-11-14T04:46:16.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9572024-11-14T04:46:16.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9582024-11-14T04:46:16.030ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9592024-11-14T04:46:16.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9602024-11-14T04:46:16.031ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Rack { parent: Fleet, key: f971eb12-0fea-4084-96d3-74198bddd289, lookup_type: ById(f971eb12-0fea-4084-96d3-74198bddd289) } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9612024-11-14T04:46:16.031ZDEBGnexus (dropshot_internal): updating rack subnet for rack f971eb12-0fea-4084-96d3-74198bddd289 to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), ) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9622024-11-14T04:46:16.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9632024-11-14T04:46:16.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9642024-11-14T04:46:16.043ZINFOnexus (dropshot_internal): Using automatic external switchport discovery actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/rack.rs:324 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9652024-11-14T04:46:16.057ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9662024-11-14T04:46:16.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9672024-11-14T04:46:16.059ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9682024-11-14T04:46:16.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9692024-11-14T04:46:16.060ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9702024-11-14T04:46:16.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9712024-11-14T04:46:16.062ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9722024-11-14T04:46:16.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000002, lookup_type: ByName("default") } result = Ok(())
9732024-11-14T04:46:16.066ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9742024-11-14T04:46:16.077ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "770fc205-ede5-4645-9607-0d5cb6cfd354", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
9752024-11-14T04:46:16.078ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 757.679461ms iteration = 1 status = {}
9762024-11-14T04:46:16.078ZDEBGnexus (DpdClient): client request actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true body = None local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9772024-11-14T04:46:16.079ZDEBGnexus (DpdClient): client response actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/ports", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f36de4a-bca6-423e-9cb3-ff176371a24b", "content-length": "24", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} }) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9782024-11-14T04:46:16.082ZINFOnexus (dropshot_internal): discovered ports for switch0: [ Internal( Internal( "int0", ), ), Rear( Rear( "rear0", ), ), Qsfp( Qsfp( "qsfp0", ), ), ] actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/rack.rs:344 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9792024-11-14T04:46:16.082ZINFOnexus (dropshot_internal): populating ports for switch0: [ Name( "qsfp0", ), ] actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/rack.rs:354 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9802024-11-14T04:46:16.083ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9812024-11-14T04:46:16.083ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9822024-11-14T04:46:16.119ZINFOnexus (dropshot_internal): Recording Rack Network Configuration actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/rack.rs:389 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9832024-11-14T04:46:16.120ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9842024-11-14T04:46:16.120ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9852024-11-14T04:46:16.156ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
9862024-11-14T04:46:16.156ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
9872024-11-14T04:46:16.156ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
9882024-11-14T04:46:16.156ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 835.620418ms
9892024-11-14T04:46:16.156ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 835.640559ms iteration = 1 status = {}
9902024-11-14T04:46:16.172ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9912024-11-14T04:46:16.172ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
9922024-11-14T04:46:16.302ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5be200f0-99fd-49a4-9718-15c5d37c684e", "content-length": "30744", "date": "Thu, 14 Nov 2024 04:46:15 GMT"} })
9932024-11-14T04:46:16.304ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
9942024-11-14T04:46:16.304ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
9952024-11-14T04:46:16.304ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
9962024-11-14T04:46:16.306ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9972024-11-14T04:46:16.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9982024-11-14T04:46:16.308ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9992024-11-14T04:46:16.310ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
10002024-11-14T04:46:16.310ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10012024-11-14T04:46:16.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
10022024-11-14T04:46:16.580ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10032024-11-14T04:46:16.581ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10042024-11-14T04:46:16.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10052024-11-14T04:46:16.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10062024-11-14T04:46:16.608ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10072024-11-14T04:46:16.608ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10082024-11-14T04:46:16.688ZDEBGnexus (dropshot_internal): failed to check if bgp peer exists in switch port settings actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true error = NotFound local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10092024-11-14T04:46:16.711ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10102024-11-14T04:46:16.712ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10112024-11-14T04:46:16.713ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10122024-11-14T04:46:16.715ZDEBGnexus (dropshot_internal): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10132024-11-14T04:46:16.715ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10142024-11-14T04:46:16.715ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10152024-11-14T04:46:16.718ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10162024-11-14T04:46:16.718ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10172024-11-14T04:46:16.720ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10182024-11-14T04:46:16.722ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = IpPool { parent: Fleet, key: 4f1c433b-f50b-467e-8ac5-368f4eee63a3, lookup_type: ByName("oxide-service-pool") } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10192024-11-14T04:46:16.722ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10202024-11-14T04:46:16.722ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10212024-11-14T04:46:16.723ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10222024-11-14T04:46:16.724ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10232024-11-14T04:46:16.725ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10242024-11-14T04:46:16.727ZDEBGnexus (dropshot_internal): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = IpPool { parent: Fleet, key: 4f1c433b-f50b-467e-8ac5-368f4eee63a3, lookup_type: ByName("oxide-service-pool") } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10252024-11-14T04:46:16.786ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10262024-11-14T04:46:16.786ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10272024-11-14T04:46:16.788ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10282024-11-14T04:46:16.790ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = BlueprintConfig result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10292024-11-14T04:46:17.094ZINFOnexus (dropshot_internal): inserted blueprint actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true blueprint_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 file = nexus/db-queries/src/db/datastore/deployment.rs:451 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10302024-11-14T04:46:17.094ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10312024-11-14T04:46:17.094ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10322024-11-14T04:46:17.096ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10332024-11-14T04:46:17.097ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = BlueprintConfig result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10342024-11-14T04:46:17.101ZINFOnexus (dropshot_internal): No networking records needed for internal_dns service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10352024-11-14T04:46:17.101ZINFOnexus (dropshot_internal): No networking records needed for internal_dns service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10362024-11-14T04:46:17.101ZINFOnexus (dropshot_internal): No networking records needed for internal_dns service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10372024-11-14T04:46:17.101ZINFOnexus (dropshot_internal): No networking records needed for cockroach_db service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10382024-11-14T04:46:17.101ZINFOnexus (dropshot_internal): No networking records needed for cockroach_db service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10392024-11-14T04:46:17.101ZINFOnexus (dropshot_internal): No networking records needed for cockroach_db service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10402024-11-14T04:46:17.101ZINFOnexus (dropshot_internal): No networking records needed for cockroach_db service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10412024-11-14T04:46:17.101ZINFOnexus (dropshot_internal): No networking records needed for cockroach_db service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10422024-11-14T04:46:17.171ZINFOnexus (dropshot_internal): Inserted networking records for external_dns service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:650 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10432024-11-14T04:46:17.187ZINFOnexus (dropshot_internal): Inserted networking records for external_dns service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:650 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10442024-11-14T04:46:17.205ZINFOnexus (dropshot_internal): Inserted networking records for nexus service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:650 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10452024-11-14T04:46:17.221ZINFOnexus (dropshot_internal): Inserted networking records for nexus service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:650 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10462024-11-14T04:46:17.237ZINFOnexus (dropshot_internal): Inserted networking records for nexus service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:650 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10472024-11-14T04:46:17.237ZINFOnexus (dropshot_internal): No networking records needed for oximeter service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10482024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for clickhouse service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10492024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible_pantry service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10502024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible_pantry service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10512024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible_pantry service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10522024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10532024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10542024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10552024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10562024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10572024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10582024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10592024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10602024-11-14T04:46:17.238ZINFOnexus (dropshot_internal): No networking records needed for crucible service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:606 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10612024-11-14T04:46:17.253ZINFOnexus (dropshot_internal): Inserted networking records for boundary_ntp service actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:650 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10622024-11-14T04:46:17.253ZINFOnexus (dropshot_internal): Inserted service networking records actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:808 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10632024-11-14T04:46:17.253ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: e7f73cb3-2603-4cca-ac59-aacb217af155, time_created: 2024-11-14T04:46:16.023612303Z, time_modified: 2024-11-14T04:46:16.023612303Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-0", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10642024-11-14T04:46:17.253ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10652024-11-14T04:46:17.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10662024-11-14T04:46:17.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10672024-11-14T04:46:17.257ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10682024-11-14T04:46:17.296ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: e2c449ea-97cc-41fd-9068-f2d1755e118e, time_created: 2024-11-14T04:46:16.023612813Z, time_modified: 2024-11-14T04:46:16.023612813Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-1", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10692024-11-14T04:46:17.296ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10702024-11-14T04:46:17.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10712024-11-14T04:46:17.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10722024-11-14T04:46:17.300ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10732024-11-14T04:46:17.309ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: 5ef2a600-e0c1-45f9-ac00-dd04915e4c6a, time_created: 2024-11-14T04:46:16.023612913Z, time_modified: 2024-11-14T04:46:16.023612913Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-2", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10742024-11-14T04:46:17.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10752024-11-14T04:46:17.309ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10762024-11-14T04:46:17.311ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10772024-11-14T04:46:17.313ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10782024-11-14T04:46:17.339ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: 5ad41b7d-910e-49cb-98f1-d37de302f53f, time_created: 2024-11-14T04:46:16.023612993Z, time_modified: 2024-11-14T04:46:16.023612993Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-3", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10792024-11-14T04:46:17.339ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10802024-11-14T04:46:17.339ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10812024-11-14T04:46:17.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10822024-11-14T04:46:17.342ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10832024-11-14T04:46:17.355ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: 6eb1eb53-a913-4c05-84d6-08c552bfadf3, time_created: 2024-11-14T04:46:16.023613073Z, time_modified: 2024-11-14T04:46:16.023613073Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-4", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10842024-11-14T04:46:17.355ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10852024-11-14T04:46:17.355ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10862024-11-14T04:46:17.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10872024-11-14T04:46:17.358ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10882024-11-14T04:46:17.371ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: 1f046e34-b8ec-45e0-9c34-86f8757336c1, time_created: 2024-11-14T04:46:16.023613143Z, time_modified: 2024-11-14T04:46:16.023613143Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-5", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10892024-11-14T04:46:17.371ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10902024-11-14T04:46:17.371ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10912024-11-14T04:46:17.373ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10922024-11-14T04:46:17.374ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10932024-11-14T04:46:17.385ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: 6fcce26e-6bff-4122-800d-46b6729919a6, time_created: 2024-11-14T04:46:16.023613223Z, time_modified: 2024-11-14T04:46:16.023613223Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-6", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10942024-11-14T04:46:17.385ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10952024-11-14T04:46:17.385ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10962024-11-14T04:46:17.394ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10972024-11-14T04:46:17.396ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10982024-11-14T04:46:17.404ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: 1c8a4219-f71e-4ed7-9c86-d8648bcd5585, time_created: 2024-11-14T04:46:16.023613303Z, time_modified: 2024-11-14T04:46:16.023613303Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-7", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
10992024-11-14T04:46:17.404ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11002024-11-14T04:46:17.404ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11012024-11-14T04:46:17.406ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11022024-11-14T04:46:17.407ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11032024-11-14T04:46:17.419ZINFOnexus (dropshot_internal): physical disk upsert in handoff: PhysicalDisk { identity: PhysicalDiskIdentity { id: 96afba8e-7682-4c33-91d3-94cf75d8ba67, time_created: 2024-11-14T04:46:16.023613373Z, time_modified: 2024-11-14T04:46:16.023613373Z, }, time_deleted: None, rcgen: Generation( Generation( 1, ), ), vendor: "synthetic-vendor", serial: "synthetic-serial-8", model: "synthetic-model-U2", variant: U2, sled_id: c073d51c-abaa-48df-abb6-2f053e9c1bec, disk_policy: InService, disk_state: Active, } actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:811 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11042024-11-14T04:46:17.419ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11052024-11-14T04:46:17.419ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11062024-11-14T04:46:17.421ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11072024-11-14T04:46:17.422ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11082024-11-14T04:46:17.430ZINFOnexus (dropshot_internal): Inserted physical disks actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:823 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11092024-11-14T04:46:17.430ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11102024-11-14T04:46:17.430ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11112024-11-14T04:46:17.431ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11122024-11-14T04:46:17.432ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11132024-11-14T04:46:17.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11142024-11-14T04:46:17.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11152024-11-14T04:46:17.442ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11162024-11-14T04:46:17.443ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11172024-11-14T04:46:17.458ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11182024-11-14T04:46:17.458ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11192024-11-14T04:46:17.459ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11202024-11-14T04:46:17.460ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11212024-11-14T04:46:17.468ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11222024-11-14T04:46:17.468ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11232024-11-14T04:46:17.470ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11242024-11-14T04:46:17.471ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11252024-11-14T04:46:17.471ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 991386bd-b06c-4985-b3ae-a10ad240ac1f file = nexus/db-queries/src/db/datastore/inventory.rs:1000
11262024-11-14T04:46:17.471ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 991386bd-b06c-4985-b3ae-a10ad240ac1f time_started = 2024-11-14 04:46:15.573353 UTC
11272024-11-14T04:46:17.471ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.151081901s iteration = 1 status = {"collection_id":"991386bd-b06c-4985-b3ae-a10ad240ac1f","time_done":"2024-11-14 04:46:16.304706 UTC","time_started":"2024-11-14 04:46:15.573353 UTC"}
11282024-11-14T04:46:17.471ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
11292024-11-14T04:46:17.471ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11302024-11-14T04:46:17.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11312024-11-14T04:46:17.473ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11322024-11-14T04:46:17.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11332024-11-14T04:46:17.474ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11342024-11-14T04:46:17.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11352024-11-14T04:46:17.476ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:79 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
11362024-11-14T04:46:17.476ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 4.517291ms iteration = 2 status = {"error":"rack not yet initialized: f971eb12-0fea-4084-96d3-74198bddd289"}
11372024-11-14T04:46:17.480ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11382024-11-14T04:46:17.480ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11392024-11-14T04:46:17.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11402024-11-14T04:46:17.482ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11412024-11-14T04:46:17.491ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11422024-11-14T04:46:17.492ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11432024-11-14T04:46:17.492ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11442024-11-14T04:46:17.493ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11452024-11-14T04:46:17.516ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11462024-11-14T04:46:17.517ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11472024-11-14T04:46:17.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11482024-11-14T04:46:17.519ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11492024-11-14T04:46:17.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11502024-11-14T04:46:17.532ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11512024-11-14T04:46:17.533ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11522024-11-14T04:46:17.534ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11532024-11-14T04:46:17.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11542024-11-14T04:46:17.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11552024-11-14T04:46:17.550ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11562024-11-14T04:46:17.551ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11572024-11-14T04:46:17.565ZINFOnexus (dropshot_internal): Inserted zpools actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:835 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11582024-11-14T04:46:17.625ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11592024-11-14T04:46:17.625ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
11602024-11-14T04:46:17.627ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11612024-11-14T04:46:17.628ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
11622024-11-14T04:46:17.629ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11632024-11-14T04:46:17.632ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: f971eb12-0fea-4084-96d3-74198bddd289, lookup_type: ById(f971eb12-0fea-4084-96d3-74198bddd289) } result = Ok(())
11642024-11-14T04:46:17.632ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: f971eb12-0fea-4084-96d3-74198bddd289, time_created: 2024-11-14T04:46:15.187444Z, time_modified: 2024-11-14T04:46:15.187444Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:768
11652024-11-14T04:46:17.941ZINFOnexus (dropshot_internal): Inserted datasets actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:865 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11662024-11-14T04:46:17.986ZINFOnexus (dropshot_internal): Populated DNS tables for internal DNS actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:876 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11672024-11-14T04:46:17.991ZINFOnexus (dropshot_internal): Populated DNS tables for external DNS actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:885 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11682024-11-14T04:46:17.991ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11692024-11-14T04:46:17.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11702024-11-14T04:46:17.994ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11712024-11-14T04:46:17.996ZDEBGnexus (dropshot_internal): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11722024-11-14T04:46:17.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11732024-11-14T04:46:17.996ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11742024-11-14T04:46:17.997ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11752024-11-14T04:46:17.999ZDEBGnexus (dropshot_internal): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11762024-11-14T04:46:18.025ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11772024-11-14T04:46:18.025ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11782024-11-14T04:46:18.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11792024-11-14T04:46:18.029ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = DnsConfig result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11802024-11-14T04:46:18.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11812024-11-14T04:46:18.029ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11822024-11-14T04:46:18.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11832024-11-14T04:46:18.033ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = DnsConfig result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11842024-11-14T04:46:18.035ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11852024-11-14T04:46:18.035ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11862024-11-14T04:46:18.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11872024-11-14T04:46:18.039ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = DnsConfig result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11882024-11-14T04:46:18.068ZINFOnexus (dropshot_internal): Created recovery silo actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:451 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11892024-11-14T04:46:18.091ZINFOnexus (dropshot_internal): Created recovery user actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:467 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11902024-11-14T04:46:18.105ZINFOnexus (dropshot_internal): Created recovery user's password actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:481 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11912024-11-14T04:46:18.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11922024-11-14T04:46:18.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11932024-11-14T04:46:18.106ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11942024-11-14T04:46:18.106ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11952024-11-14T04:46:18.108ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11962024-11-14T04:46:18.109ZDEBGnexus (dropshot_internal): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11972024-11-14T04:46:18.109ZDEBGnexus (dropshot_internal): Generated role assignment queries actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11982024-11-14T04:46:18.113ZINFOnexus (dropshot_internal): Granted Silo privileges actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/db-queries/src/db/datastore/rack.rs:509 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
11992024-11-14T04:46:18.113ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12002024-11-14T04:46:18.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12012024-11-14T04:46:18.115ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12022024-11-14T04:46:18.116ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12032024-11-14T04:46:18.153ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12042024-11-14T04:46:18.154ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12052024-11-14T04:46:18.155ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12062024-11-14T04:46:18.155ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12072024-11-14T04:46:18.156ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12082024-11-14T04:46:18.156ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12092024-11-14T04:46:18.158ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12102024-11-14T04:46:18.158ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12112024-11-14T04:46:18.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12122024-11-14T04:46:18.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12132024-11-14T04:46:18.161ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12142024-11-14T04:46:18.161ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12152024-11-14T04:46:18.162ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12162024-11-14T04:46:18.165ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12172024-11-14T04:46:18.165ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12182024-11-14T04:46:18.165ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12192024-11-14T04:46:18.166ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12202024-11-14T04:46:18.167ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12212024-11-14T04:46:18.168ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12222024-11-14T04:46:18.168ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12232024-11-14T04:46:18.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12242024-11-14T04:46:18.172ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12252024-11-14T04:46:18.172ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12262024-11-14T04:46:18.172ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12272024-11-14T04:46:18.176ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12282024-11-14T04:46:18.176ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12292024-11-14T04:46:18.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12302024-11-14T04:46:18.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12312024-11-14T04:46:18.178ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12322024-11-14T04:46:18.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12332024-11-14T04:46:18.180ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12342024-11-14T04:46:18.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12352024-11-14T04:46:18.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12362024-11-14T04:46:18.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12372024-11-14T04:46:18.182ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12382024-11-14T04:46:18.183ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12392024-11-14T04:46:18.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12402024-11-14T04:46:18.187ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12412024-11-14T04:46:18.187ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12422024-11-14T04:46:18.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12432024-11-14T04:46:18.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12442024-11-14T04:46:18.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12452024-11-14T04:46:18.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12462024-11-14T04:46:18.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12472024-11-14T04:46:18.191ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12482024-11-14T04:46:18.191ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12492024-11-14T04:46:18.193ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12502024-11-14T04:46:18.193ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12512024-11-14T04:46:18.195ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12522024-11-14T04:46:18.195ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12532024-11-14T04:46:18.196ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12542024-11-14T04:46:18.196ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12552024-11-14T04:46:18.197ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12562024-11-14T04:46:18.200ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12572024-11-14T04:46:18.200ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12582024-11-14T04:46:18.201ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12592024-11-14T04:46:18.202ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12602024-11-14T04:46:18.202ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12612024-11-14T04:46:18.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12622024-11-14T04:46:18.203ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12632024-11-14T04:46:18.204ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12642024-11-14T04:46:18.208ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12652024-11-14T04:46:18.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12662024-11-14T04:46:18.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12672024-11-14T04:46:18.211ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12682024-11-14T04:46:18.211ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12692024-11-14T04:46:18.213ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12702024-11-14T04:46:18.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12712024-11-14T04:46:18.214ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12722024-11-14T04:46:18.214ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12732024-11-14T04:46:18.216ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12742024-11-14T04:46:18.216ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12752024-11-14T04:46:18.218ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12762024-11-14T04:46:18.218ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12772024-11-14T04:46:18.219ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12782024-11-14T04:46:18.219ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12792024-11-14T04:46:18.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12802024-11-14T04:46:18.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12812024-11-14T04:46:18.221ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12822024-11-14T04:46:18.226ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12832024-11-14T04:46:18.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12842024-11-14T04:46:18.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12852024-11-14T04:46:18.227ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12862024-11-14T04:46:18.227ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12872024-11-14T04:46:18.229ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12882024-11-14T04:46:18.229ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12892024-11-14T04:46:18.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12902024-11-14T04:46:18.233ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12912024-11-14T04:46:18.233ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12922024-11-14T04:46:18.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12932024-11-14T04:46:18.240ZDEBGnexus: resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 32de9250-63c9-46be-9a6b-ee4af19ab1ab, kind: Service { id: 48221b8a-d832-4d05-ae47-e92f81388c0c }, name: Name("external-dns-48221b8a-d832-4d05-ae47-e92f81388c0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 126])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: da2f10b4-11b4-4931-a7f7-d415353256ec, kind: Service { id: 8c0d54f6-76b3-47cc-9828-b8e677cc2961 }, name: Name("external-dns-8c0d54f6-76b3-47cc-9828-b8e677cc2961"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 129])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 68cb12a7-f65f-4940-ad7a-12ac0e8fa220, kind: Service { id: 7d1b51fe-adf1-4350-9c25-9240054e3911 }, name: Name("nexus-7d1b51fe-adf1-4350-9c25-9240054e3911"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 137, 76])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5946ca8-a389-4a86-aed2-97d4e8639c67, kind: Service { id: 13db289d-f9b6-41e9-b3bd-5ab4f3105694 }, name: Name("nexus-13db289d-f9b6-41e9-b3bd-5ab4f3105694"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b897d57d-b436-467c-89e9-441e43e441fa, kind: Service { id: 79994e02-6747-4071-83af-7872b31640fb }, name: Name("nexus-79994e02-6747-4071-83af-7872b31640fb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 66])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
12942024-11-14T04:46:18.240ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12952024-11-14T04:46:18.240ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12962024-11-14T04:46:18.241ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12972024-11-14T04:46:18.243ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Fleet result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12982024-11-14T04:46:18.243ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b roles = RoleSet { roles: {} } uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
12992024-11-14T04:46:18.243ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b resource = Database result = Ok(()) uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
13002024-11-14T04:46:18.244ZDEBGnexus: Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
13012024-11-14T04:46:18.247ZDEBGnexus: resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 32de9250-63c9-46be-9a6b-ee4af19ab1ab, kind: Service { id: 48221b8a-d832-4d05-ae47-e92f81388c0c }, name: Name("external-dns-48221b8a-d832-4d05-ae47-e92f81388c0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 126])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: da2f10b4-11b4-4931-a7f7-d415353256ec, kind: Service { id: 8c0d54f6-76b3-47cc-9828-b8e677cc2961 }, name: Name("external-dns-8c0d54f6-76b3-47cc-9828-b8e677cc2961"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 129])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 68cb12a7-f65f-4940-ad7a-12ac0e8fa220, kind: Service { id: 7d1b51fe-adf1-4350-9c25-9240054e3911 }, name: Name("nexus-7d1b51fe-adf1-4350-9c25-9240054e3911"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 137, 76])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5946ca8-a389-4a86-aed2-97d4e8639c67, kind: Service { id: 13db289d-f9b6-41e9-b3bd-5ab4f3105694 }, name: Name("nexus-13db289d-f9b6-41e9-b3bd-5ab4f3105694"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b897d57d-b436-467c-89e9-441e43e441fa, kind: Service { id: 79994e02-6747-4071-83af-7872b31640fb }, name: Name("nexus-79994e02-6747-4071-83af-7872b31640fb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 66])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
13022024-11-14T04:46:18.247ZDEBGnexus: resolved 2 rules for sleds
13032024-11-14T04:46:18.272ZDEBGnexus: resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: c073d51c-abaa-48df-abb6-2f053e9c1bec, time_created: 2024-11-14T04:46:13.572725Z, time_modified: 2024-11-14T04:46:13.572725Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: f971eb12-0fea-4084-96d3-74198bddd289, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
13042024-11-14T04:46:18.272ZDEBGnexus: sending firewall rules to sled agents
13052024-11-14T04:46:18.272ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
13062024-11-14T04:46:18.273ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
13072024-11-14T04:46:18.275ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
13082024-11-14T04:46:18.275ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
13092024-11-14T04:46:18.276ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
13102024-11-14T04:46:18.278ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
13112024-11-14T04:46:18.308ZDEBGnexus: client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
13122024-11-14T04:46:18.310ZDEBGnexus: client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "077ca60c-1f33-49b5-9493-5a0536d38cd5", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
13132024-11-14T04:46:18.311ZINFOnexus: updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:496
13142024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
13152024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
13162024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
13172024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
13182024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
13192024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13202024-11-14T04:46:18.311ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2292206 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:34258 req_id = 6beafc38-f87e-48fa-bad0-2235b2685f6b response_code = 204 uri = /racks/f971eb12-0fea-4084-96d3-74198bddd289/initialization-complete
13212024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
13222024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13232024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
13242024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
13252024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
13262024-11-14T04:46:18.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13272024-11-14T04:46:18.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13282024-11-14T04:46:18.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13292024-11-14T04:46:18.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
13302024-11-14T04:46:18.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13312024-11-14T04:46:18.312ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 957, tv_nsec: 84511278 } })
13322024-11-14T04:46:18.312ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 957, tv_nsec: 84681743 } })
13332024-11-14T04:46:18.312ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
13342024-11-14T04:46:18.312ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.06217ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
13352024-11-14T04:46:18.313ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
13362024-11-14T04:46:18.313ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.256437ms iteration = 2 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
13372024-11-14T04:46:18.313ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13382024-11-14T04:46:18.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13392024-11-14T04:46:18.313ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13402024-11-14T04:46:18.313ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13412024-11-14T04:46:18.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13422024-11-14T04:46:18.314ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13432024-11-14T04:46:18.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13442024-11-14T04:46:18.316ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13452024-11-14T04:46:18.316ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13462024-11-14T04:46:18.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13472024-11-14T04:46:18.316ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
13482024-11-14T04:46:18.316ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
13492024-11-14T04:46:18.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
13502024-11-14T04:46:18.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13512024-11-14T04:46:18.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13522024-11-14T04:46:18.319ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(991386bd-b06c-4985-b3ae-a10ad240ac1f, 0), (4af73f1a-e5f2-4f3a-a815-42b664dfa0ef, 0), (600ec60a-2dd6-472c-b285-024765cf6c90, 0), (5f2ddd08-a8e8-4775-819d-824daf062f34, 0)] collection_id = 991386bd-b06c-4985-b3ae-a10ad240ac1f
13532024-11-14T04:46:18.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
13542024-11-14T04:46:18.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13552024-11-14T04:46:18.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
13562024-11-14T04:46:18.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13572024-11-14T04:46:18.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13582024-11-14T04:46:18.321ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13592024-11-14T04:46:18.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
13602024-11-14T04:46:18.325ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13612024-11-14T04:46:18.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13622024-11-14T04:46:18.348ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
13632024-11-14T04:46:18.348ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
13642024-11-14T04:46:18.348ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
13652024-11-14T04:46:18.348ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13662024-11-14T04:46:18.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13672024-11-14T04:46:18.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13682024-11-14T04:46:18.351ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
13692024-11-14T04:46:18.351ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
13702024-11-14T04:46:18.351ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
13712024-11-14T04:46:18.351ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13722024-11-14T04:46:18.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13732024-11-14T04:46:18.352ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13742024-11-14T04:46:18.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13752024-11-14T04:46:18.353ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13762024-11-14T04:46:18.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13772024-11-14T04:46:18.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13782024-11-14T04:46:18.355ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13792024-11-14T04:46:18.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13802024-11-14T04:46:18.360ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13812024-11-14T04:46:18.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13822024-11-14T04:46:18.362ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13832024-11-14T04:46:18.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13842024-11-14T04:46:18.364ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13852024-11-14T04:46:18.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13862024-11-14T04:46:18.375ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
13872024-11-14T04:46:18.375ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
13882024-11-14T04:46:18.375ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13892024-11-14T04:46:18.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13902024-11-14T04:46:18.377ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13912024-11-14T04:46:18.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13922024-11-14T04:46:18.379ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13932024-11-14T04:46:18.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13942024-11-14T04:46:18.420ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
13952024-11-14T04:46:18.420ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 108.681518ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
13962024-11-14T04:46:18.421ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
13972024-11-14T04:46:18.421ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
13982024-11-14T04:46:18.421ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13992024-11-14T04:46:18.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14002024-11-14T04:46:18.423ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14012024-11-14T04:46:18.424ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
14022024-11-14T04:46:18.424ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
14032024-11-14T04:46:18.424ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
14042024-11-14T04:46:18.424ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 112.627481ms iteration = 2 status = {"generation":1}
14052024-11-14T04:46:18.424ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
14062024-11-14T04:46:18.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14072024-11-14T04:46:18.425ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14082024-11-14T04:46:18.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14092024-11-14T04:46:18.455ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
14102024-11-14T04:46:18.455ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
14112024-11-14T04:46:18.455ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
14122024-11-14T04:46:18.455ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 143.273059ms iteration = 2 status = {"generation":2}
14132024-11-14T04:46:18.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
14142024-11-14T04:46:18.455ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
14152024-11-14T04:46:18.467ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 991386bd-b06c-4985-b3ae-a10ad240ac1f file = nexus/db-queries/src/db/datastore/inventory.rs:1446 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 84 nerrors = 0 nnics = 6 nnvme_disk_firmware = 11 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agent_zones = 1 nsled_agents = 1 nsps = 2 nzones = 28 nzpools = 9
14162024-11-14T04:46:18.467ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14172024-11-14T04:46:18.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14182024-11-14T04:46:18.479ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(4af73f1a-e5f2-4f3a-a815-42b664dfa0ef, 0), (600ec60a-2dd6-472c-b285-024765cf6c90, 0), (5f2ddd08-a8e8-4775-819d-824daf062f34, 0)]
14192024-11-14T04:46:18.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 957, tv_nsec: 252575015 } })
14202024-11-14T04:46:18.533ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
14212024-11-14T04:46:18.535ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
14222024-11-14T04:46:18.565ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
14232024-11-14T04:46:18.565ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
14242024-11-14T04:46:18.565ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
14252024-11-14T04:46:18.569ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "b619a7e6-d10b-4820-b246-b51e9571c225", "content-length": "476", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14262024-11-14T04:46:18.569ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
14272024-11-14T04:46:18.570ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec7cd68c-ff0d-4d6a-abbe-a9bde3b9f963", "content-length": "734", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14282024-11-14T04:46:18.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
14292024-11-14T04:46:18.570ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "82174292-7fdc-4a41-8c68-1b2166bb4e16", "content-length": "111", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14302024-11-14T04:46:18.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
14312024-11-14T04:46:18.570ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8bcf6b5-13a4-481c-a336-3fe56e1d7938", "content-length": "111", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14322024-11-14T04:46:18.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
14332024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "917f3f5b-837b-493d-84b7-492fbb58fd16", "content-length": "167", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14342024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
14352024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4f71db1-31fb-48ac-85c3-ec2c67959630", "content-length": "167", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14362024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
14372024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "172fc66b-7957-4203-be8e-bc09abb74155", "content-length": "175", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14382024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
14392024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c822123-874a-4ede-935f-5660ff2ff702", "content-length": "175", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14402024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
14412024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "84a72d10-7d1f-408c-8c44-2f2677d877cd", "content-length": "702", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14422024-11-14T04:46:18.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14432024-11-14T04:46:18.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09f5ead3-6986-4610-bea6-88366e3fea73", "content-length": "727", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14442024-11-14T04:46:18.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14452024-11-14T04:46:18.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09cce210-2c3f-4cb6-9434-bc8959b5036c", "content-length": "729", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14462024-11-14T04:46:18.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14472024-11-14T04:46:18.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebfbaf13-94ee-4b2b-a5d2-c138b3915ae3", "content-length": "728", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14482024-11-14T04:46:18.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
14492024-11-14T04:46:18.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2da729c5-8b49-42a7-8dc8-1901763141b1", "content-length": "722", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14502024-11-14T04:46:18.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
14512024-11-14T04:46:18.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e8f59f7-a9de-4ea8-9f12-16aa1018158a", "content-length": "109", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14522024-11-14T04:46:18.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
14532024-11-14T04:46:18.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23a69b1e-10cc-4018-8a8c-c90bc61411ec", "content-length": "109", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14542024-11-14T04:46:18.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
14552024-11-14T04:46:18.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eaa3204-0d7c-4e3b-a63b-2b6339d4bd35", "content-length": "169", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14562024-11-14T04:46:18.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
14572024-11-14T04:46:18.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2c72954-1fdf-4fc3-87c4-7ab2b142533f", "content-length": "169", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14582024-11-14T04:46:18.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
14592024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e40b4502-4369-4b2d-8c9f-b623bd0610c9", "content-length": "178", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14602024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
14612024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e881af3e-4a60-4772-bd9e-d6aa45c249df", "content-length": "178", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14622024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
14632024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcb12c8d-2d8c-4dc4-a224-8598ef4e676d", "content-length": "702", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14642024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
14652024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e273cf8-094e-4608-9bc2-3d3db5bb6ed2", "content-length": "727", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14662024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
14672024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e23c832-35a2-4861-a9da-4c8c2ab234d2", "content-length": "729", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14682024-11-14T04:46:18.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
14692024-11-14T04:46:18.575ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66adbdfd-5e33-4dbc-9e08-b36f2ec61af8", "content-length": "728", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14702024-11-14T04:46:18.575ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14712024-11-14T04:46:18.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14722024-11-14T04:46:18.577ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14732024-11-14T04:46:18.578ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
14742024-11-14T04:46:18.578ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14752024-11-14T04:46:18.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14762024-11-14T04:46:18.580ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14772024-11-14T04:46:18.582ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
14782024-11-14T04:46:18.582ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14792024-11-14T04:46:18.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14802024-11-14T04:46:18.610ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
14812024-11-14T04:46:18.610ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
14822024-11-14T04:46:18.644ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "2667d597-ffa3-4e11-b070-1265abc3e5b1", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
14832024-11-14T04:46:18.644ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
14842024-11-14T04:46:18.647ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "243f4932-d648-487a-a127-9b369a7e21f1", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
14852024-11-14T04:46:18.647ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
14862024-11-14T04:46:18.650ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "94a0b3b4-94e0-48f7-a116-a1cf3947a44a", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
14872024-11-14T04:46:18.650ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
14882024-11-14T04:46:18.651ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
14892024-11-14T04:46:18.652ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 227.412116ms iteration = 3 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
14902024-11-14T04:46:18.654ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "4377134c-be11-4f10-a1c3-5e74f99e7c1a", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14912024-11-14T04:46:18.654ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14922024-11-14T04:46:18.656ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "735e51d1-89f3-48fe-a24e-e0b3ccbca7ef", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14932024-11-14T04:46:18.656ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14942024-11-14T04:46:18.658ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14952024-11-14T04:46:18.658ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 202.829374ms iteration = 3 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
14962024-11-14T04:46:18.732ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
14972024-11-14T04:46:18.732ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
14982024-11-14T04:46:18.944ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc6bb811-e6ed-43a9-88ac-bd6c181b1b3e", "content-length": "30744", "date": "Thu, 14 Nov 2024 04:46:18 GMT"} })
14992024-11-14T04:46:18.946ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
15002024-11-14T04:46:18.946ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
15012024-11-14T04:46:18.946ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
15022024-11-14T04:46:18.948ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15032024-11-14T04:46:18.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
15042024-11-14T04:46:18.950ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15052024-11-14T04:46:18.952ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
15062024-11-14T04:46:18.952ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15072024-11-14T04:46:18.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
15082024-11-14T04:46:19.495ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 677476f2-8740-4b58-93e4-82e4fac86fd6 file = nexus/db-queries/src/db/datastore/inventory.rs:1000
15092024-11-14T04:46:19.495ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 677476f2-8740-4b58-93e4-82e4fac86fd6 time_started = 2024-11-14 04:46:18.565543 UTC
15102024-11-14T04:46:19.495ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.183170023s iteration = 2 status = {"collection_id":"677476f2-8740-4b58-93e4-82e4fac86fd6","time_done":"2024-11-14 04:46:18.946648 UTC","time_started":"2024-11-14 04:46:18.565543 UTC"}
15112024-11-14T04:46:19.495ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Dependency
15122024-11-14T04:46:19.495ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15132024-11-14T04:46:19.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15142024-11-14T04:46:19.496ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15152024-11-14T04:46:19.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15162024-11-14T04:46:19.498ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15172024-11-14T04:46:19.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15182024-11-14T04:46:19.500ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15192024-11-14T04:46:19.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15202024-11-14T04:46:19.502ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15212024-11-14T04:46:19.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15222024-11-14T04:46:19.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15232024-11-14T04:46:19.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15242024-11-14T04:46:19.604ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 109.537383ms iteration = 3 status = {"physical_disks_added":0}
15252024-11-14T04:46:19.633ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
15262024-11-14T04:46:19.633ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
15272024-11-14T04:46:19.634ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
15282024-11-14T04:46:19.634ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
15292024-11-14T04:46:19.636ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15302024-11-14T04:46:19.638ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: f971eb12-0fea-4084-96d3-74198bddd289, lookup_type: ById(f971eb12-0fea-4084-96d3-74198bddd289) } result = Ok(())
15312024-11-14T04:46:19.638ZINFOnexus: Rack initialized file = nexus/src/app/rack.rs:765
15322024-11-14T04:46:19.638ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15332024-11-14T04:46:19.638ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15342024-11-14T04:46:19.640ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15352024-11-14T04:46:19.640ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15362024-11-14T04:46:19.641ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15372024-11-14T04:46:19.641ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15382024-11-14T04:46:19.643ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15392024-11-14T04:46:19.643ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15402024-11-14T04:46:19.644ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15412024-11-14T04:46:19.644ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15422024-11-14T04:46:19.645ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15432024-11-14T04:46:19.645ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15442024-11-14T04:46:19.646ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15452024-11-14T04:46:19.649ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
15462024-11-14T04:46:19.649ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15472024-11-14T04:46:19.650ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15482024-11-14T04:46:19.651ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15492024-11-14T04:46:19.651ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15502024-11-14T04:46:19.652ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15512024-11-14T04:46:19.652ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15522024-11-14T04:46:19.654ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15532024-11-14T04:46:19.656ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
15542024-11-14T04:46:19.656ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15552024-11-14T04:46:19.657ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15562024-11-14T04:46:19.658ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15572024-11-14T04:46:19.658ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15582024-11-14T04:46:19.660ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15592024-11-14T04:46:19.660ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15602024-11-14T04:46:19.661ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15612024-11-14T04:46:19.661ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15622024-11-14T04:46:19.662ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15632024-11-14T04:46:19.662ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15642024-11-14T04:46:19.663ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15652024-11-14T04:46:19.664ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15662024-11-14T04:46:19.664ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15672024-11-14T04:46:19.665ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15682024-11-14T04:46:19.666ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15692024-11-14T04:46:19.669ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
15702024-11-14T04:46:19.669ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15712024-11-14T04:46:19.669ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15722024-11-14T04:46:19.670ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15732024-11-14T04:46:19.670ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15742024-11-14T04:46:19.671ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15752024-11-14T04:46:19.672ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15762024-11-14T04:46:19.674ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15772024-11-14T04:46:19.674ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15782024-11-14T04:46:19.675ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15792024-11-14T04:46:19.675ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15802024-11-14T04:46:19.676ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15812024-11-14T04:46:19.677ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15822024-11-14T04:46:19.678ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15832024-11-14T04:46:19.678ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15842024-11-14T04:46:19.679ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15852024-11-14T04:46:19.685ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15862024-11-14T04:46:19.685ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15872024-11-14T04:46:19.685ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15882024-11-14T04:46:19.686ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15892024-11-14T04:46:19.686ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15902024-11-14T04:46:19.687ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15912024-11-14T04:46:19.687ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15922024-11-14T04:46:19.689ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15932024-11-14T04:46:19.692ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15942024-11-14T04:46:19.692ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15952024-11-14T04:46:19.692ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15962024-11-14T04:46:19.698ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15972024-11-14T04:46:19.698ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15982024-11-14T04:46:19.700ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
15992024-11-14T04:46:19.700ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16002024-11-14T04:46:19.701ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16012024-11-14T04:46:19.701ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16022024-11-14T04:46:19.703ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16032024-11-14T04:46:19.703ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16042024-11-14T04:46:19.704ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16052024-11-14T04:46:19.705ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16062024-11-14T04:46:19.706ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16072024-11-14T04:46:19.706ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16082024-11-14T04:46:19.707ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16092024-11-14T04:46:19.707ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16102024-11-14T04:46:19.708ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16112024-11-14T04:46:19.712ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16122024-11-14T04:46:19.712ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16132024-11-14T04:46:19.712ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16142024-11-14T04:46:19.713ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16152024-11-14T04:46:19.713ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16162024-11-14T04:46:19.714ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16172024-11-14T04:46:19.714ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16182024-11-14T04:46:19.715ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16192024-11-14T04:46:19.720ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16202024-11-14T04:46:19.720ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16212024-11-14T04:46:19.720ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16222024-11-14T04:46:19.726ZDEBGnexus (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 32de9250-63c9-46be-9a6b-ee4af19ab1ab, kind: Service { id: 48221b8a-d832-4d05-ae47-e92f81388c0c }, name: Name("external-dns-48221b8a-d832-4d05-ae47-e92f81388c0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 126])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: da2f10b4-11b4-4931-a7f7-d415353256ec, kind: Service { id: 8c0d54f6-76b3-47cc-9828-b8e677cc2961 }, name: Name("external-dns-8c0d54f6-76b3-47cc-9828-b8e677cc2961"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 129])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 68cb12a7-f65f-4940-ad7a-12ac0e8fa220, kind: Service { id: 7d1b51fe-adf1-4350-9c25-9240054e3911 }, name: Name("nexus-7d1b51fe-adf1-4350-9c25-9240054e3911"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 137, 76])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5946ca8-a389-4a86-aed2-97d4e8639c67, kind: Service { id: 13db289d-f9b6-41e9-b3bd-5ab4f3105694 }, name: Name("nexus-13db289d-f9b6-41e9-b3bd-5ab4f3105694"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b897d57d-b436-467c-89e9-441e43e441fa, kind: Service { id: 79994e02-6747-4071-83af-7872b31640fb }, name: Name("nexus-79994e02-6747-4071-83af-7872b31640fb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 66])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
16232024-11-14T04:46:19.726ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16242024-11-14T04:46:19.726ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16252024-11-14T04:46:19.728ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16262024-11-14T04:46:19.729ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16272024-11-14T04:46:19.729ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16282024-11-14T04:46:19.729ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16292024-11-14T04:46:19.766ZDEBGnexus (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
16302024-11-14T04:46:19.766ZDEBGnexus (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 32de9250-63c9-46be-9a6b-ee4af19ab1ab, kind: Service { id: 48221b8a-d832-4d05-ae47-e92f81388c0c }, name: Name("external-dns-48221b8a-d832-4d05-ae47-e92f81388c0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 126])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: da2f10b4-11b4-4931-a7f7-d415353256ec, kind: Service { id: 8c0d54f6-76b3-47cc-9828-b8e677cc2961 }, name: Name("external-dns-8c0d54f6-76b3-47cc-9828-b8e677cc2961"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 129])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 68cb12a7-f65f-4940-ad7a-12ac0e8fa220, kind: Service { id: 7d1b51fe-adf1-4350-9c25-9240054e3911 }, name: Name("nexus-7d1b51fe-adf1-4350-9c25-9240054e3911"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 137, 76])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5946ca8-a389-4a86-aed2-97d4e8639c67, kind: Service { id: 13db289d-f9b6-41e9-b3bd-5ab4f3105694 }, name: Name("nexus-13db289d-f9b6-41e9-b3bd-5ab4f3105694"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b897d57d-b436-467c-89e9-441e43e441fa, kind: Service { id: 79994e02-6747-4071-83af-7872b31640fb }, name: Name("nexus-79994e02-6747-4071-83af-7872b31640fb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 66])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
16312024-11-14T04:46:19.766ZDEBGnexus (InternalApi): resolved 2 rules for sleds
16322024-11-14T04:46:19.795ZDEBGnexus (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: c073d51c-abaa-48df-abb6-2f053e9c1bec, time_created: 2024-11-14T04:46:13.572725Z, time_modified: 2024-11-14T04:46:13.572725Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: f971eb12-0fea-4084-96d3-74198bddd289, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
16332024-11-14T04:46:19.795ZDEBGnexus (InternalApi): sending firewall rules to sled agents
16342024-11-14T04:46:19.795ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16352024-11-14T04:46:19.796ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16362024-11-14T04:46:19.808ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
16372024-11-14T04:46:19.809ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16382024-11-14T04:46:19.810ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16392024-11-14T04:46:19.813ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
16402024-11-14T04:46:19.844ZDEBGnexus (InternalApi): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16412024-11-14T04:46:19.844ZDEBGnexus (InternalApi): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "2386d2cd-bdb3-46d8-b625-b29fe218a142", "date": "Thu, 14 Nov 2024 04:46:19 GMT"} })
16422024-11-14T04:46:19.845ZINFOnexus (InternalApi): updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:496
16432024-11-14T04:46:19.845ZINFOnexus: plumbed initial IP allowlist file = nexus/src/app/allow_list.rs:161
16442024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = / versions = all
16452024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /access versions = all
16462024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /assets/{path} versions = all
16472024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /device/auth versions = all
16482024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /device/confirm versions = all
16492024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /device/success versions = all
16502024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /device/token versions = all
16512024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /device/verify versions = all
16522024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /experimental/v1/probes versions = all
16532024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /experimental/v1/probes versions = all
16542024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
16552024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /experimental/v1/probes/{probe} versions = all
16562024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /images versions = all
16572024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /login versions = all
16582024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /login/{silo_name}/local versions = all
16592024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
16602024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
16612024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
16622024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /lookup/{path} versions = all
16632024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /projects/{path} versions = all
16642024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /projects-new versions = all
16652024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /settings/{path} versions = all
16662024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /system/{path} versions = all
16672024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /utilization versions = all
16682024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/certificates versions = all
16692024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/certificates versions = all
16702024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/certificates/{certificate} versions = all
16712024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/certificates/{certificate} versions = all
16722024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/disks versions = all
16732024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks versions = all
16742024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/disks/{disk} versions = all
16752024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/disks/{disk} versions = all
16762024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
16772024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
16782024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
16792024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks/{disk}/finalize versions = all
16802024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
16812024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/floating-ips versions = all
16822024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/floating-ips versions = all
16832024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
16842024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
16852024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
16862024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
16872024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
16882024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/groups versions = all
16892024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/groups/{group_id} versions = all
16902024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/images versions = all
16912024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/images versions = all
16922024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/images/{image} versions = all
16932024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/images/{image} versions = all
16942024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/images/{image}/demote versions = all
16952024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/images/{image}/promote versions = all
16962024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances versions = all
16972024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances versions = all
16982024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/instances/{instance} versions = all
16992024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance} versions = all
17002024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/instances/{instance} versions = all
17012024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/disks versions = all
17022024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all
17032024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all
17042024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
17052024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
17062024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
17072024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/reboot versions = all
17082024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
17092024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
17102024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
17112024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/start versions = all
17122024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/stop versions = all
17132024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
17142024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
17152024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
17162024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
17172024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
17182024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
17192024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/internet-gateways versions = all
17202024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/internet-gateways versions = all
17212024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
17222024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
17232024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/ip-pools versions = all
17242024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/ip-pools/{pool} versions = all
17252024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/login/{silo_name}/local versions = all
17262024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/logout versions = all
17272024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/me versions = all
17282024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/me/groups versions = all
17292024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/me/ssh-keys versions = all
17302024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/me/ssh-keys versions = all
17312024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
17322024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
17332024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/metrics/{metric_name} versions = all
17342024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/network-interfaces versions = all
17352024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/network-interfaces versions = all
17362024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
17372024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/network-interfaces/{interface} versions = all
17382024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
17392024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/ping versions = all
17402024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/policy versions = all
17412024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/policy versions = all
17422024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/projects versions = all
17432024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/projects versions = all
17442024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/projects/{project} versions = all
17452024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/projects/{project} versions = all
17462024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/projects/{project} versions = all
17472024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/projects/{project}/policy versions = all
17482024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/projects/{project}/policy versions = all
17492024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/snapshots versions = all
17502024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/snapshots versions = all
17512024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
17522024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/snapshots/{snapshot} versions = all
17532024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/disks versions = all
17542024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
17552024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/racks versions = all
17562024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
17572024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds versions = all
17582024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/hardware/sleds versions = all
17592024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
17602024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
17612024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
17622024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
17632024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
17642024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/switch-port versions = all
17652024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
17662024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
17672024-11-14T04:46:19.862ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
17682024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/switches versions = all
17692024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
17702024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/identity-providers versions = all
17712024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/identity-providers/local/users versions = all
17722024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
17732024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
17742024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/identity-providers/saml versions = all
17752024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
17762024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools versions = all
17772024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools versions = all
17782024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
17792024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
17802024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
17812024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
17822024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
17832024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
17842024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
17852024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
17862024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
17872024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
17882024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
17892024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools-service versions = all
17902024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
17912024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
17922024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
17932024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
17942024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/address-lot versions = all
17952024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/address-lot versions = all
17962024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
17972024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
17982024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/allow-list versions = all
17992024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/networking/allow-list versions = all
18002024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/bfd-disable versions = all
18012024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/bfd-enable versions = all
18022024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bfd-status versions = all
18032024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/bgp versions = all
18042024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp versions = all
18052024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/bgp versions = all
18062024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
18072024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
18082024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
18092024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
18102024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-exported versions = all
18112024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
18122024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
18132024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-status versions = all
18142024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/loopback-address versions = all
18152024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/loopback-address versions = all
18162024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
18172024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
18182024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
18192024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
18202024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
18212024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/policy versions = all
18222024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/policy versions = all
18232024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/roles versions = all
18242024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/roles/{role_name} versions = all
18252024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silo-quotas versions = all
18262024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos versions = all
18272024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/silos versions = all
18282024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/silos/{silo} versions = all
18292024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos/{silo} versions = all
18302024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
18312024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
18322024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
18332024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
18342024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
18352024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/timeseries/query versions = all
18362024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/timeseries/schemas versions = all
18372024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/update/repository versions = all
18382024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/update/repository/{system_version} versions = all
18392024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/users versions = all
18402024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/users/{user_id} versions = all
18412024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/users-builtin versions = all
18422024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/users-builtin/{user} versions = all
18432024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/utilization/silos versions = all
18442024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
18452024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/users versions = all
18462024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/utilization versions = all
18472024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-firewall-rules versions = all
18482024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpc-firewall-rules versions = all
18492024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-router-routes versions = all
18502024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/vpc-router-routes versions = all
18512024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
18522024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
18532024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
18542024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-routers versions = all
18552024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/vpc-routers versions = all
18562024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
18572024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-routers/{router} versions = all
18582024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpc-routers/{router} versions = all
18592024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-subnets versions = all
18602024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/vpc-subnets versions = all
18612024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
18622024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
18632024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
18642024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
18652024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpcs versions = all
18662024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/vpcs versions = all
18672024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
18682024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpcs/{vpc} versions = all
18692024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpcs/{vpc} versions = all
18702024-11-14T04:46:19.863ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = 172.30.2.6:443
18712024-11-14T04:46:19.863ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.6:443
18722024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = / versions = all
18732024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /access versions = all
18742024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /assets/{path} versions = all
18752024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /device/auth versions = all
18762024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /device/confirm versions = all
18772024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /device/success versions = all
18782024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /device/token versions = all
18792024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /device/verify versions = all
18802024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /experimental/v1/probes versions = all
18812024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /experimental/v1/probes versions = all
18822024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
18832024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
18842024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /images versions = all
18852024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /login versions = all
18862024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /login/{silo_name}/local versions = all
18872024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
18882024-11-14T04:46:19.867ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
18892024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
18902024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /lookup/{path} versions = all
18912024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /projects/{path} versions = all
18922024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /projects-new versions = all
18932024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /settings/{path} versions = all
18942024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /system/{path} versions = all
18952024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /utilization versions = all
18962024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/certificates versions = all
18972024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/certificates versions = all
18982024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
18992024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/certificates/{certificate} versions = all
19002024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/disks versions = all
19012024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks versions = all
19022024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/disks/{disk} versions = all
19032024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/disks/{disk} versions = all
19042024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
19052024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
19062024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
19072024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
19082024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
19092024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/floating-ips versions = all
19102024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/floating-ips versions = all
19112024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
19122024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
19132024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
19142024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
19152024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
19162024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/groups versions = all
19172024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/groups/{group_id} versions = all
19182024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/images versions = all
19192024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/images versions = all
19202024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/images/{image} versions = all
19212024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/images/{image} versions = all
19222024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/images/{image}/demote versions = all
19232024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/images/{image}/promote versions = all
19242024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances versions = all
19252024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances versions = all
19262024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/instances/{instance} versions = all
19272024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance} versions = all
19282024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/instances/{instance} versions = all
19292024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
19302024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
19312024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
19322024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
19332024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
19342024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
19352024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
19362024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
19372024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
19382024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
19392024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/start versions = all
19402024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
19412024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
19422024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
19432024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
19442024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
19452024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
19462024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
19472024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/internet-gateways versions = all
19482024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/internet-gateways versions = all
19492024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
19502024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
19512024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/ip-pools versions = all
19522024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
19532024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
19542024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/logout versions = all
19552024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/me versions = all
19562024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/me/groups versions = all
19572024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/me/ssh-keys versions = all
19582024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/me/ssh-keys versions = all
19592024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
19602024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
19612024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
19622024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/network-interfaces versions = all
19632024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/network-interfaces versions = all
19642024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
19652024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
19662024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
19672024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/ping versions = all
19682024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/policy versions = all
19692024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/policy versions = all
19702024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/projects versions = all
19712024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/projects versions = all
19722024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/projects/{project} versions = all
19732024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/projects/{project} versions = all
19742024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/projects/{project} versions = all
19752024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/projects/{project}/policy versions = all
19762024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
19772024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/snapshots versions = all
19782024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/snapshots versions = all
19792024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
19802024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
19812024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/disks versions = all
19822024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
19832024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/racks versions = all
19842024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
19852024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds versions = all
19862024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/hardware/sleds versions = all
19872024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
19882024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
19892024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
19902024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
19912024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
19922024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
19932024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
19942024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
19952024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
19962024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/switches versions = all
19972024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
19982024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/identity-providers versions = all
19992024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
20002024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
20012024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
20022024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
20032024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
20042024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools versions = all
20052024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools versions = all
20062024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
20072024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
20082024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
20092024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
20102024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
20112024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
20122024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
20132024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
20142024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
20152024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
20162024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
20172024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools-service versions = all
20182024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
20192024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
20202024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
20212024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
20222024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/address-lot versions = all
20232024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/address-lot versions = all
20242024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
20252024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
20262024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/allow-list versions = all
20272024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
20282024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
20292024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
20302024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
20312024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
20322024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp versions = all
20332024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/bgp versions = all
20342024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
20352024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
20362024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
20372024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
20382024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
20392024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
20402024-11-14T04:46:19.868ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
20412024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
20422024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
20432024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
20442024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
20452024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
20462024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
20472024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
20482024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
20492024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/policy versions = all
20502024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/policy versions = all
20512024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/roles versions = all
20522024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/roles/{role_name} versions = all
20532024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silo-quotas versions = all
20542024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos versions = all
20552024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/silos versions = all
20562024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
20572024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos/{silo} versions = all
20582024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
20592024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
20602024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
20612024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
20622024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
20632024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/timeseries/query versions = all
20642024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
20652024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/update/repository versions = all
20662024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/update/repository/{system_version} versions = all
20672024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/users versions = all
20682024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/users/{user_id} versions = all
20692024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/users-builtin versions = all
20702024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
20712024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/utilization/silos versions = all
20722024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
20732024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/users versions = all
20742024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/utilization versions = all
20752024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
20762024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
20772024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-router-routes versions = all
20782024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/vpc-router-routes versions = all
20792024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
20802024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
20812024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
20822024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-routers versions = all
20832024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/vpc-routers versions = all
20842024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
20852024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
20862024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
20872024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-subnets versions = all
20882024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/vpc-subnets versions = all
20892024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
20902024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
20912024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
20922024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
20932024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpcs versions = all
20942024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/vpcs versions = all
20952024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
20962024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
20972024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
20982024-11-14T04:46:19.869ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::b]:12228
20992024-11-14T04:46:19.869ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::b]:12228
21002024-11-14T04:46:19.869ZDEBGnexus (nexus-producer-server): registered DTrace probes
21012024-11-14T04:46:19.872ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::b]:49209 method = GET path = /{producer_id} versions = all
21022024-11-14T04:46:19.873ZINFOnexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::b]:49209
21032024-11-14T04:46:19.873ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::b]:49209
21042024-11-14T04:46:19.873ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::b]:49209
21052024-11-14T04:46:19.873ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::b]:12221
21062024-11-14T04:46:19.873ZDEBGnexus (nexus-producer-server): starting producer registration task
21072024-11-14T04:46:19.874ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::b]:49209 file = oximeter/producer/src/lib.rs:283 interval = 10s producer_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
21082024-11-14T04:46:19.874ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
21092024-11-14T04:46:19.874ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
21102024-11-14T04:46:19.905ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
21112024-11-14T04:46:19.905ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:63588
21122024-11-14T04:46:19.907ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:63588 req_id = 126ded87-2fbd-483a-b7b8-f7c811f1270b roles = RoleSet { roles: {} } uri = /metrics/producers
21132024-11-14T04:46:19.907ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:63588 req_id = 126ded87-2fbd-483a-b7b8-f7c811f1270b resource = Database result = Ok(()) uri = /metrics/producers
21142024-11-14T04:46:19.953ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
21152024-11-14T04:46:19.953ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47668 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:63588 req_id = 126ded87-2fbd-483a-b7b8-f7c811f1270b response_code = 201 uri = /metrics/producers
21162024-11-14T04:46:19.953ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "126ded87-2fbd-483a-b7b8-f7c811f1270b", "content-length": "41", "date": "Thu, 14 Nov 2024 04:46:19 GMT"} })
21172024-11-14T04:46:19.954ZDEBGnexus (producer-registration-task): registered with nexus successfully
21182024-11-14T04:46:19.954ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
21192024-11-14T04:46:20.389ZDEBGnexus (NexusCertResolver): resolved TLS certificate certificate = TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" } server_name = recovery.sys.oxide.test silo_id = 992dae43-7740-49b5-814b-c5341916fc63
21202024-11-14T04:46:20.397ZINFOnexus (dropshot_external): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = 172.30.2.6:443 remote_addr = 10.151.2.100:34303
21212024-11-14T04:46:20.409ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21222024-11-14T04:46:20.410ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21232024-11-14T04:46:20.414ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21242024-11-14T04:46:20.414ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21252024-11-14T04:46:20.416ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21262024-11-14T04:46:20.416ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21272024-11-14T04:46:20.418ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21282024-11-14T04:46:20.419ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ByName("recovery") } result = Ok(())
21292024-11-14T04:46:20.419ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21302024-11-14T04:46:20.419ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21312024-11-14T04:46:20.421ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21322024-11-14T04:46:20.421ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21332024-11-14T04:46:20.422ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21342024-11-14T04:46:20.425ZDEBGnexus (ExternalAuthn): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ByName("recovery") } result = Ok(())
21352024-11-14T04:46:20.425ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21362024-11-14T04:46:20.425ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21372024-11-14T04:46:20.433ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21382024-11-14T04:46:20.433ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21392024-11-14T04:46:20.435ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21402024-11-14T04:46:20.435ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21412024-11-14T04:46:20.437ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21422024-11-14T04:46:20.438ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ByName("recovery") }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ByName("recovery") } result = Ok(())
21432024-11-14T04:46:20.438ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21442024-11-14T04:46:20.438ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21452024-11-14T04:46:21.150ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21462024-11-14T04:46:21.150ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21472024-11-14T04:46:21.151ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21482024-11-14T04:46:21.152ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21492024-11-14T04:46:21.154ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21502024-11-14T04:46:21.154ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21512024-11-14T04:46:21.156ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21522024-11-14T04:46:21.156ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21532024-11-14T04:46:21.158ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21542024-11-14T04:46:21.159ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
21552024-11-14T04:46:21.159ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21562024-11-14T04:46:21.159ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21572024-11-14T04:46:21.161ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21582024-11-14T04:46:21.162ZDEBGnexus (ExternalAuthn): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSessionList result = Ok(())
21592024-11-14T04:46:21.162ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21602024-11-14T04:46:21.162ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21612024-11-14T04:46:21.165ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 759618 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:34303 req_id = 453774c7-ffae-452c-8320-e97b6c18bc59 response_code = 204 uri = /v1/login/recovery/local
21622024-11-14T04:46:21.204ZDEBGnexus (NexusCertResolver): resolved TLS certificate certificate = TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" } server_name = recovery.sys.oxide.test silo_id = 992dae43-7740-49b5-814b-c5341916fc63
21632024-11-14T04:46:21.209ZINFOnexus (dropshot_external): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = 172.30.2.6:443 remote_addr = 10.151.2.100:52982
21642024-11-14T04:46:21.210ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21652024-11-14T04:46:21.210ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21662024-11-14T04:46:21.218ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21672024-11-14T04:46:21.218ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21682024-11-14T04:46:21.220ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21692024-11-14T04:46:21.221ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
21702024-11-14T04:46:21.221ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21712024-11-14T04:46:21.221ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21722024-11-14T04:46:21.224ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21732024-11-14T04:46:21.224ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21742024-11-14T04:46:21.226ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21752024-11-14T04:46:21.226ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21762024-11-14T04:46:21.227ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21772024-11-14T04:46:21.227ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21782024-11-14T04:46:21.229ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21792024-11-14T04:46:21.230ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
21802024-11-14T04:46:21.230ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21812024-11-14T04:46:21.230ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21822024-11-14T04:46:21.232ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21832024-11-14T04:46:21.233ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
21842024-11-14T04:46:21.233ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21852024-11-14T04:46:21.233ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21862024-11-14T04:46:21.243ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21872024-11-14T04:46:21.243ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21882024-11-14T04:46:21.251ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21892024-11-14T04:46:21.251ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21902024-11-14T04:46:21.252ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21912024-11-14T04:46:21.252ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21922024-11-14T04:46:21.253ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21932024-11-14T04:46:21.253ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21942024-11-14T04:46:21.255ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
21952024-11-14T04:46:21.256ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
21962024-11-14T04:46:21.256ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21972024-11-14T04:46:21.256ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
21982024-11-14T04:46:21.257ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
21992024-11-14T04:46:21.258ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22002024-11-14T04:46:21.259ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22012024-11-14T04:46:21.259ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22022024-11-14T04:46:21.260ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22032024-11-14T04:46:21.261ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
22042024-11-14T04:46:21.261ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c roles = RoleSet { roles: {} } uri = /v1/projects
22052024-11-14T04:46:21.261ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c resource = Database result = Ok(()) uri = /v1/projects
22062024-11-14T04:46:21.271ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c roles = RoleSet { roles: {} } uri = /v1/projects
22072024-11-14T04:46:21.271ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c resource = Database result = Ok(()) uri = /v1/projects
22082024-11-14T04:46:21.280ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c roles = RoleSet { roles: {} } uri = /v1/projects
22092024-11-14T04:46:21.280ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c resource = Database result = Ok(()) uri = /v1/projects
22102024-11-14T04:46:21.308ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/projects
22112024-11-14T04:46:21.310ZDEBGnexus (dropshot_external): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(()) uri = /v1/projects
22122024-11-14T04:46:21.310ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c roles = RoleSet { roles: {} } uri = /v1/projects
22132024-11-14T04:46:21.310ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c resource = Database result = Ok(()) uri = /v1/projects
22142024-11-14T04:46:21.313ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103779 local_addr = 172.30.2.6:443 method = GET remote_addr = 10.151.2.100:52982 req_id = 552f0d09-8c8e-4273-89f6-64e44a4b129c response_code = 200 uri = /v1/projects
22152024-11-14T04:46:21.318ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22162024-11-14T04:46:21.318ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22172024-11-14T04:46:21.320ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22182024-11-14T04:46:21.320ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22192024-11-14T04:46:21.321ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22202024-11-14T04:46:21.322ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
22212024-11-14T04:46:21.323ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22222024-11-14T04:46:21.323ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22232024-11-14T04:46:21.325ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22242024-11-14T04:46:21.325ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22252024-11-14T04:46:21.327ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22262024-11-14T04:46:21.327ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22272024-11-14T04:46:21.328ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22282024-11-14T04:46:21.328ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22292024-11-14T04:46:21.329ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22302024-11-14T04:46:21.331ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
22312024-11-14T04:46:21.331ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22322024-11-14T04:46:21.331ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22332024-11-14T04:46:21.332ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22342024-11-14T04:46:21.333ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
22352024-11-14T04:46:21.333ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22362024-11-14T04:46:21.333ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22372024-11-14T04:46:21.348ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22382024-11-14T04:46:21.349ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22392024-11-14T04:46:21.350ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22402024-11-14T04:46:21.350ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22412024-11-14T04:46:21.351ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22422024-11-14T04:46:21.352ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22432024-11-14T04:46:21.353ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22442024-11-14T04:46:21.353ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22452024-11-14T04:46:21.354ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22462024-11-14T04:46:21.356ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
22472024-11-14T04:46:21.356ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22482024-11-14T04:46:21.356ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22492024-11-14T04:46:21.357ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22502024-11-14T04:46:21.357ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22512024-11-14T04:46:21.359ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22522024-11-14T04:46:21.359ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22532024-11-14T04:46:21.360ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22542024-11-14T04:46:21.361ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
22552024-11-14T04:46:21.361ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools
22562024-11-14T04:46:21.361ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 resource = Database result = Ok(()) uri = /v1/system/ip-pools
22572024-11-14T04:46:21.370ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools
22582024-11-14T04:46:21.371ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 resource = Database result = Ok(()) uri = /v1/system/ip-pools
22592024-11-14T04:46:21.373ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/system/ip-pools
22602024-11-14T04:46:21.375ZDEBGnexus (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 resource = IpPoolList result = Ok(()) uri = /v1/system/ip-pools
22612024-11-14T04:46:21.375ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools
22622024-11-14T04:46:21.375ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 resource = Database result = Ok(()) uri = /v1/system/ip-pools
22632024-11-14T04:46:21.378ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60311 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 0003529d-ea2f-46f4-86d7-32219e50ef22 response_code = 201 uri = /v1/system/ip-pools
22642024-11-14T04:46:21.381ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22652024-11-14T04:46:21.381ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22662024-11-14T04:46:21.383ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22672024-11-14T04:46:21.383ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22682024-11-14T04:46:21.384ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22692024-11-14T04:46:21.385ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
22702024-11-14T04:46:21.386ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22712024-11-14T04:46:21.386ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22722024-11-14T04:46:21.387ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22732024-11-14T04:46:21.387ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22742024-11-14T04:46:21.388ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22752024-11-14T04:46:21.388ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22762024-11-14T04:46:21.390ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22772024-11-14T04:46:21.390ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22782024-11-14T04:46:21.391ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22792024-11-14T04:46:21.393ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
22802024-11-14T04:46:21.393ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22812024-11-14T04:46:21.393ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22822024-11-14T04:46:21.394ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22832024-11-14T04:46:21.395ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
22842024-11-14T04:46:21.395ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22852024-11-14T04:46:21.396ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22862024-11-14T04:46:21.399ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22872024-11-14T04:46:21.399ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22882024-11-14T04:46:21.401ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22892024-11-14T04:46:21.401ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22902024-11-14T04:46:21.402ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22912024-11-14T04:46:21.403ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22922024-11-14T04:46:21.404ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22932024-11-14T04:46:21.404ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22942024-11-14T04:46:21.406ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
22952024-11-14T04:46:21.407ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
22962024-11-14T04:46:21.407ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22972024-11-14T04:46:21.407ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
22982024-11-14T04:46:21.408ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22992024-11-14T04:46:21.408ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23002024-11-14T04:46:21.409ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23012024-11-14T04:46:21.410ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23022024-11-14T04:46:21.411ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
23032024-11-14T04:46:21.412ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
23042024-11-14T04:46:21.412ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23052024-11-14T04:46:21.412ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23062024-11-14T04:46:21.414ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23072024-11-14T04:46:21.414ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23082024-11-14T04:46:21.424ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23092024-11-14T04:46:21.425ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23102024-11-14T04:46:21.426ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/system/ip-pools/default/silos
23112024-11-14T04:46:21.428ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = IpPool { parent: Fleet, key: 33988e00-bb76-4174-bd3d-7d8a44e2aa22, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/silos
23122024-11-14T04:46:21.428ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23132024-11-14T04:46:21.429ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23142024-11-14T04:46:21.439ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23152024-11-14T04:46:21.439ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23162024-11-14T04:46:21.442ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23172024-11-14T04:46:21.442ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23182024-11-14T04:46:21.445ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23192024-11-14T04:46:21.445ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23202024-11-14T04:46:21.448ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23212024-11-14T04:46:21.448ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23222024-11-14T04:46:21.449ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/system/ip-pools/default/silos
23232024-11-14T04:46:21.451ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ByName("recovery") } result = Ok(()) uri = /v1/system/ip-pools/default/silos
23242024-11-14T04:46:21.451ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23252024-11-14T04:46:21.451ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23262024-11-14T04:46:21.454ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23272024-11-14T04:46:21.454ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23282024-11-14T04:46:21.457ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/system/ip-pools/default/silos
23292024-11-14T04:46:21.459ZDEBGnexus (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = IpPoolList result = Ok(()) uri = /v1/system/ip-pools/default/silos
23302024-11-14T04:46:21.459ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
23312024-11-14T04:46:21.459ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
23322024-11-14T04:46:21.510ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129244 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = c3fa9b63-514d-4331-ac3c-57ff00e285c3 response_code = 201 uri = /v1/system/ip-pools/default/silos
23332024-11-14T04:46:21.512ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23342024-11-14T04:46:21.512ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23352024-11-14T04:46:21.521ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23362024-11-14T04:46:21.521ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23372024-11-14T04:46:21.522ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
23382024-11-14T04:46:21.524ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
23392024-11-14T04:46:21.524ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23402024-11-14T04:46:21.524ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23412024-11-14T04:46:21.525ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23422024-11-14T04:46:21.525ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23432024-11-14T04:46:21.527ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23442024-11-14T04:46:21.527ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23452024-11-14T04:46:21.529ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23462024-11-14T04:46:21.529ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23472024-11-14T04:46:21.530ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
23482024-11-14T04:46:21.532ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
23492024-11-14T04:46:21.532ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23502024-11-14T04:46:21.532ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23512024-11-14T04:46:21.533ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
23522024-11-14T04:46:21.534ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
23532024-11-14T04:46:21.534ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23542024-11-14T04:46:21.534ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23552024-11-14T04:46:21.538ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23562024-11-14T04:46:21.538ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23572024-11-14T04:46:21.539ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23582024-11-14T04:46:21.540ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23592024-11-14T04:46:21.541ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23602024-11-14T04:46:21.541ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23612024-11-14T04:46:21.543ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23622024-11-14T04:46:21.543ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23632024-11-14T04:46:21.544ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
23642024-11-14T04:46:21.545ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
23652024-11-14T04:46:21.545ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23662024-11-14T04:46:21.545ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23672024-11-14T04:46:21.547ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23682024-11-14T04:46:21.547ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23692024-11-14T04:46:21.548ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23702024-11-14T04:46:21.548ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23712024-11-14T04:46:21.550ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
23722024-11-14T04:46:21.551ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
23732024-11-14T04:46:21.551ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
23742024-11-14T04:46:21.551ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23752024-11-14T04:46:21.560ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
23762024-11-14T04:46:21.560ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23772024-11-14T04:46:21.562ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
23782024-11-14T04:46:21.562ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23792024-11-14T04:46:21.564ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/system/ip-pools/default/ranges/add
23802024-11-14T04:46:21.566ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = IpPool { parent: Fleet, key: 33988e00-bb76-4174-bd3d-7d8a44e2aa22, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23812024-11-14T04:46:21.566ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
23822024-11-14T04:46:21.566ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23832024-11-14T04:46:21.568ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
23842024-11-14T04:46:21.568ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23852024-11-14T04:46:21.569ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
23862024-11-14T04:46:21.569ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23872024-11-14T04:46:21.571ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
23882024-11-14T04:46:21.571ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23892024-11-14T04:46:21.573ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/system/ip-pools/default/ranges/add
23902024-11-14T04:46:21.574ZDEBGnexus (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead resource = IpPool { parent: Fleet, key: 33988e00-bb76-4174-bd3d-7d8a44e2aa22, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
23912024-11-14T04:46:21.591ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80196 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = ed89eb64-b580-47b4-a78e-7e71611b0ead response_code = 201 uri = /v1/system/ip-pools/default/ranges/add
23922024-11-14T04:46:21.593ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23932024-11-14T04:46:21.593ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23942024-11-14T04:46:21.595ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23952024-11-14T04:46:21.595ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
23962024-11-14T04:46:21.596ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
23972024-11-14T04:46:21.598ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
23982024-11-14T04:46:21.598ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
23992024-11-14T04:46:21.598ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24002024-11-14T04:46:21.599ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24012024-11-14T04:46:21.599ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24022024-11-14T04:46:21.600ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24032024-11-14T04:46:21.601ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24042024-11-14T04:46:21.602ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24052024-11-14T04:46:21.602ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24062024-11-14T04:46:21.604ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24072024-11-14T04:46:21.605ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
24082024-11-14T04:46:21.605ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24092024-11-14T04:46:21.605ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24102024-11-14T04:46:21.606ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24112024-11-14T04:46:21.607ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
24122024-11-14T04:46:21.607ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24132024-11-14T04:46:21.607ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24142024-11-14T04:46:21.612ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24152024-11-14T04:46:21.612ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24162024-11-14T04:46:21.616ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24172024-11-14T04:46:21.616ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24182024-11-14T04:46:21.617ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24192024-11-14T04:46:21.618ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24202024-11-14T04:46:21.619ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24212024-11-14T04:46:21.619ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24222024-11-14T04:46:21.621ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24232024-11-14T04:46:21.622ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
24242024-11-14T04:46:21.622ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24252024-11-14T04:46:21.622ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24262024-11-14T04:46:21.623ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24272024-11-14T04:46:21.624ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24282024-11-14T04:46:21.625ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24292024-11-14T04:46:21.625ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24302024-11-14T04:46:21.626ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24312024-11-14T04:46:21.628ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
24322024-11-14T04:46:21.628ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {} } uri = /v1/system/silos/recovery/quotas
24332024-11-14T04:46:21.628ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Database result = Ok(()) uri = /v1/system/silos/recovery/quotas
24342024-11-14T04:46:21.629ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {} } uri = /v1/system/silos/recovery/quotas
24352024-11-14T04:46:21.629ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Database result = Ok(()) uri = /v1/system/silos/recovery/quotas
24362024-11-14T04:46:21.632ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {} } uri = /v1/system/silos/recovery/quotas
24372024-11-14T04:46:21.632ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Database result = Ok(()) uri = /v1/system/silos/recovery/quotas
24382024-11-14T04:46:21.634ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {} } uri = /v1/system/silos/recovery/quotas
24392024-11-14T04:46:21.635ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Database result = Ok(()) uri = /v1/system/silos/recovery/quotas
24402024-11-14T04:46:21.637ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/system/silos/recovery/quotas
24412024-11-14T04:46:21.638ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ByName("recovery") } result = Ok(()) uri = /v1/system/silos/recovery/quotas
24422024-11-14T04:46:21.638ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {} } uri = /v1/system/silos/recovery/quotas
24432024-11-14T04:46:21.638ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Database result = Ok(()) uri = /v1/system/silos/recovery/quotas
24442024-11-14T04:46:21.640ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {} } uri = /v1/system/silos/recovery/quotas
24452024-11-14T04:46:21.640ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Database result = Ok(()) uri = /v1/system/silos/recovery/quotas
24462024-11-14T04:46:21.643ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {} } uri = /v1/system/silos/recovery/quotas
24472024-11-14T04:46:21.643ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Database result = Ok(()) uri = /v1/system/silos/recovery/quotas
24482024-11-14T04:46:21.645ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/system/silos/recovery/quotas
24492024-11-14T04:46:21.646ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ByName("recovery") } result = Ok(()) uri = /v1/system/silos/recovery/quotas
24502024-11-14T04:46:21.646ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 roles = RoleSet { roles: {} } uri = /v1/system/silos/recovery/quotas
24512024-11-14T04:46:21.646ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 resource = Database result = Ok(()) uri = /v1/system/silos/recovery/quotas
24522024-11-14T04:46:21.650ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57739 local_addr = 172.30.2.6:443 method = PUT remote_addr = 10.151.2.100:52982 req_id = 1e86d033-546b-4c9a-a9ca-493c1f4a8483 response_code = 200 uri = /v1/system/silos/recovery/quotas
24532024-11-14T04:46:21.653ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24542024-11-14T04:46:21.654ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24552024-11-14T04:46:21.655ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24562024-11-14T04:46:21.655ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24572024-11-14T04:46:21.656ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24582024-11-14T04:46:21.657ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
24592024-11-14T04:46:21.657ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24602024-11-14T04:46:21.657ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24612024-11-14T04:46:21.659ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24622024-11-14T04:46:21.659ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24632024-11-14T04:46:21.660ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24642024-11-14T04:46:21.660ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24652024-11-14T04:46:21.662ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24662024-11-14T04:46:21.662ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24672024-11-14T04:46:21.663ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24682024-11-14T04:46:21.664ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
24692024-11-14T04:46:21.664ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24702024-11-14T04:46:21.664ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24712024-11-14T04:46:21.665ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24722024-11-14T04:46:21.666ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
24732024-11-14T04:46:21.666ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24742024-11-14T04:46:21.666ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24752024-11-14T04:46:21.670ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24762024-11-14T04:46:21.670ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24772024-11-14T04:46:21.671ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24782024-11-14T04:46:21.671ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24792024-11-14T04:46:21.673ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24802024-11-14T04:46:21.673ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24812024-11-14T04:46:21.674ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24822024-11-14T04:46:21.675ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24832024-11-14T04:46:21.675ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24842024-11-14T04:46:21.677ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
24852024-11-14T04:46:21.677ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24862024-11-14T04:46:21.677ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24872024-11-14T04:46:21.678ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24882024-11-14T04:46:21.678ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24892024-11-14T04:46:21.680ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
24902024-11-14T04:46:21.680ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
24912024-11-14T04:46:21.682ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
24922024-11-14T04:46:21.683ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
24932024-11-14T04:46:21.683ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab roles = RoleSet { roles: {} } uri = /v1/projects
24942024-11-14T04:46:21.683ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab resource = Database result = Ok(()) uri = /v1/projects
24952024-11-14T04:46:21.685ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab roles = RoleSet { roles: {} } uri = /v1/projects
24962024-11-14T04:46:21.686ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab resource = Database result = Ok(()) uri = /v1/projects
24972024-11-14T04:46:21.688ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab roles = RoleSet { roles: {} } uri = /v1/projects
24982024-11-14T04:46:21.688ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab resource = Database result = Ok(()) uri = /v1/projects
24992024-11-14T04:46:21.690ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/projects
25002024-11-14T04:46:21.692ZDEBGnexus (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(()) uri = /v1/projects
25012024-11-14T04:46:21.693ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create
25022024-11-14T04:46:21.693ZINFOnexus (SEC): saga create dag = {"end_node":19,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[8,9,null],[9,10,null],[10,11,null],[11,12,null],[12,13,null],[13,14,null],[14,15,null],[15,16,null],[2,3,null],[16,17,null],[18,0,null],[17,19,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"project_create.project_create_record","label":"ProjectCreateRecord","name":"project"}},{"Action":{"action_name":"project_create.project_create_vpc_params","label":"ProjectCreateVpcParams","name":"vpc_create_params"}},{"SubsagaStart":{"params_node_name":"vpc_create_params","saga_name":"vpc-create"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateVpcId","name":"vpc_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateSystemRouterId","name":"system_router_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateDefaultV4RouteId","name":"default_v4_route_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateDefaultV6RouteId","name":"default_v6_route_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateDefaultSubnetId","name":"default_subnet_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateDefaultInternetGatewayId","name":"default_internet_gateway_id"}},{"Action":{"action_name":"vpc_create.vpc_create_vpc","label":"VpcCreateVpc","name":"vpc"}},{"Action":{"action_name":"vpc_create.vpc_create_router","label":"VpcCreateRouter","name":"router"}},{"Action":{"action_name":"vpc_create.vpc_create_v4_route","label":"VpcCreateV4Route","name":"route4"}},{"Action":{"action_name":"vpc_create.vpc_create_v6_route","label":"VpcCreateV6Route","name":"route6"}},{"Action":{"action_name":"vpc_create.vpc_create_subnet","label":"VpcCreateSubnet","name":"subnet"}},{"Action":{"action_name":"vpc_create.vpc_update_firewall","label":"VpcUpdateFirewall","name":"firewall"}},{"Action":{"action_name":"vpc_create.vpc_create_gateway","label":"VpcCreateGateway","name":"gateway"}},{"Action":{"action_name":"vpc_create.vpc_notify_sleds","label":"VpcNotifySleds","name":"no_result"}},{"SubsagaEnd":{"name":"vpc"}},{"Start":{"params":{"authz_silo":{"key":"992dae43-7740-49b5-814b-c5341916fc63","lookup_type":{"ById":"992dae43-7740-49b5-814b-c5341916fc63"},"parent":null},"project_create":{"description":"","name":"proj-a32f5c8c37"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","silo_user_id":"7d466341-8999-4ad7-a241-ecff7d0e71de"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"project-create","start_node":18} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25032024-11-14T04:46:21.693ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create
25042024-11-14T04:46:21.730ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create
25052024-11-14T04:46:21.730ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25062024-11-14T04:46:21.730ZDEBGnexus (SEC): saga log event new_state = N018 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25072024-11-14T04:46:21.730ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 18 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25082024-11-14T04:46:21.742ZDEBGnexus (SEC): saga log event new_state = N018 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25092024-11-14T04:46:21.742ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 18 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25102024-11-14T04:46:21.755ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25112024-11-14T04:46:21.755ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25122024-11-14T04:46:21.765ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25132024-11-14T04:46:21.765ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25142024-11-14T04:46:21.768ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25152024-11-14T04:46:21.768ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25162024-11-14T04:46:21.771ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25172024-11-14T04:46:21.771ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25182024-11-14T04:46:21.775ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25192024-11-14T04:46:21.776ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25202024-11-14T04:46:21.776ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25212024-11-14T04:46:21.776ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = ProjectCreateRecord
25222024-11-14T04:46:21.794ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25232024-11-14T04:46:21.794ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Array [Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}, Object {"identity": Object {"description": String(""), "id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "name": String("proj-a32f5c8c37"), "time_created": String("2024-11-14T04:46:21.765323Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:21.765323Z")}, "rcgen": Number(1), "silo_id": String("992dae43-7740-49b5-814b-c5341916fc63")}]) node_id = 0 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25242024-11-14T04:46:21.798ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25252024-11-14T04:46:21.798ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25262024-11-14T04:46:21.809ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25272024-11-14T04:46:21.809ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"authz_project": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}, "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("992dae43-7740-49b5-814b-c5341916fc63"), "silo_user_id": String("7d466341-8999-4ad7-a241-ecff7d0e71de")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "vpc_create": Object {"description": String("Default VPC"), "dns_name": String("default"), "ipv6_prefix": String("fde3:3072:2af8::/48"), "name": String("default")}}) node_id = 1 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25282024-11-14T04:46:21.820ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25292024-11-14T04:46:21.820ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25302024-11-14T04:46:21.825ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25312024-11-14T04:46:21.825ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25322024-11-14T04:46:21.829ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25332024-11-14T04:46:21.829ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25342024-11-14T04:46:21.833ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25352024-11-14T04:46:21.833ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")) node_id = 3 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25362024-11-14T04:46:21.836ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25372024-11-14T04:46:21.836ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25382024-11-14T04:46:21.840ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25392024-11-14T04:46:21.840ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768")) node_id = 4 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25402024-11-14T04:46:21.844ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25412024-11-14T04:46:21.844ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25422024-11-14T04:46:21.848ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25432024-11-14T04:46:21.848ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("75dc1454-5a63-4592-b544-903cc9e95db0")) node_id = 5 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25442024-11-14T04:46:21.851ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25452024-11-14T04:46:21.851ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25462024-11-14T04:46:21.855ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25472024-11-14T04:46:21.855ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("3b514609-bf26-4103-b8a9-4a162e45f55a")) node_id = 6 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25482024-11-14T04:46:21.859ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25492024-11-14T04:46:21.859ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25502024-11-14T04:46:21.863ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25512024-11-14T04:46:21.863ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("64123868-fdd8-44b4-af58-803d800a258a")) node_id = 7 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25522024-11-14T04:46:21.867ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25532024-11-14T04:46:21.867ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25542024-11-14T04:46:21.872ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25552024-11-14T04:46:21.872ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("6af791b4-74b7-406f-b55b-25de6525abe8")) node_id = 8 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25562024-11-14T04:46:21.875ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25572024-11-14T04:46:21.875ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25582024-11-14T04:46:21.879ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25592024-11-14T04:46:21.879ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25602024-11-14T04:46:21.882ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25612024-11-14T04:46:21.882ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25622024-11-14T04:46:21.885ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25632024-11-14T04:46:21.885ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25642024-11-14T04:46:21.887ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25652024-11-14T04:46:21.888ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25662024-11-14T04:46:21.890ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25672024-11-14T04:46:21.892ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25682024-11-14T04:46:21.892ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25692024-11-14T04:46:21.893ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateVpc
25702024-11-14T04:46:21.906ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25712024-11-14T04:46:21.907ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Array [Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}, Object {"dns_name": String("default"), "firewall_gen": Number(1), "identity": Object {"description": String("Default VPC"), "id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "name": String("default"), "time_created": String("2024-11-14T04:46:21.879483Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:21.879483Z")}, "ipv6_prefix": String("fde3:3072:2af8::/48"), "project_id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "subnet_gen": Number(1), "system_router_id": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768"), "vni": Number(11383694)}]) node_id = 9 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25722024-11-14T04:46:21.911ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25732024-11-14T04:46:21.911ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25742024-11-14T04:46:21.918ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25752024-11-14T04:46:21.918ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25762024-11-14T04:46:21.920ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25772024-11-14T04:46:21.920ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25782024-11-14T04:46:21.923ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25792024-11-14T04:46:21.923ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25802024-11-14T04:46:21.925ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25812024-11-14T04:46:21.925ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25822024-11-14T04:46:21.927ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25832024-11-14T04:46:21.930ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25842024-11-14T04:46:21.930ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25852024-11-14T04:46:21.930ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateRouter
25862024-11-14T04:46:21.970ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25872024-11-14T04:46:21.970ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"key": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768"), "lookup_type": Object {"ById": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}) node_id = 10 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25882024-11-14T04:46:21.974ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
25892024-11-14T04:46:21.974ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
25902024-11-14T04:46:21.982ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25912024-11-14T04:46:21.982ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25922024-11-14T04:46:21.984ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25932024-11-14T04:46:21.985ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25942024-11-14T04:46:21.987ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25952024-11-14T04:46:21.987ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25962024-11-14T04:46:21.990ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25972024-11-14T04:46:21.990ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25982024-11-14T04:46:21.992ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
25992024-11-14T04:46:21.995ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") }, key: fc9cb0b5-8ba0-4acb-8aa7-669d702de768, lookup_type: ById(fc9cb0b5-8ba0-4acb-8aa7-669d702de768) } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
26002024-11-14T04:46:21.995ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
26012024-11-14T04:46:21.995ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV4Route
26022024-11-14T04:46:22.005ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26032024-11-14T04:46:22.005ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26042024-11-14T04:46:22.009ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26052024-11-14T04:46:22.009ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26062024-11-14T04:46:22.012ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26072024-11-14T04:46:22.012ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26082024-11-14T04:46:22.015ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26092024-11-14T04:46:22.015ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26102024-11-14T04:46:22.017ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26112024-11-14T04:46:22.017ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26122024-11-14T04:46:22.019ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26132024-11-14T04:46:22.020ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26142024-11-14T04:46:22.022ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26152024-11-14T04:46:22.025ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") }, key: fc9cb0b5-8ba0-4acb-8aa7-669d702de768, lookup_type: ById(fc9cb0b5-8ba0-4acb-8aa7-669d702de768) } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26162024-11-14T04:46:22.025ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26172024-11-14T04:46:22.025ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateV6Route
26182024-11-14T04:46:22.036ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26192024-11-14T04:46:22.036ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26202024-11-14T04:46:22.039ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26212024-11-14T04:46:22.039ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26222024-11-14T04:46:22.042ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26232024-11-14T04:46:22.042ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26242024-11-14T04:46:22.044ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26252024-11-14T04:46:22.044ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26262024-11-14T04:46:22.047ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26272024-11-14T04:46:22.047ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26282024-11-14T04:46:22.050ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26292024-11-14T04:46:22.050ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26302024-11-14T04:46:22.052ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26312024-11-14T04:46:22.055ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26322024-11-14T04:46:22.064ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26332024-11-14T04:46:22.064ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26342024-11-14T04:46:22.076ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26352024-11-14T04:46:22.076ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateSubnet
26362024-11-14T04:46:22.079ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26372024-11-14T04:46:22.079ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Array [Object {"key": String("64123868-fdd8-44b4-af58-803d800a258a"), "lookup_type": Object {"ById": String("64123868-fdd8-44b4-af58-803d800a258a")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}, Object {"custom_router_id": Null, "identity": Object {"description": String("The default subnet for default"), "id": String("64123868-fdd8-44b4-af58-803d800a258a"), "name": String("default"), "time_created": String("2024-11-14T04:46:22.042084Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.042084Z")}, "ipv4_block": String("172.30.0.0/22"), "ipv6_block": String("fde3:3072:2af8::/64"), "rcgen": Number(1), "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}]) node_id = 13 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26382024-11-14T04:46:22.083ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26392024-11-14T04:46:22.084ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26402024-11-14T04:46:22.087ZDEBGnexus: default firewall rules for vpc default rules = [VpcFirewallRule { identity: VpcFirewallRuleIdentity { id: f867c751-a27a-4a13-8cca-8e2aaadbb702, name: Name(Name("allow-internal-inbound")), description: "allow inbound traffic to all instances within the VPC if originated within the VPC", time_created: 2024-11-14T04:46:22.087447399Z, time_modified: 2024-11-14T04:46:22.087447399Z, time_deleted: None }, vpc_id: dea2096d-dbe9-4ffb-9b54-01f8106501ad, status: VpcFirewallRuleStatus(Enabled), direction: VpcFirewallRuleDirection(Inbound), targets: [VpcFirewallRuleTarget(Vpc(Name("default")))], filter_hosts: Some([VpcFirewallRuleHostFilter(Vpc(Name("default")))]), filter_ports: None, filter_protocols: None, action: VpcFirewallRuleAction(Allow), priority: VpcFirewallRulePriority(VpcFirewallRulePriority(65534)) }, VpcFirewallRule { identity: VpcFirewallRuleIdentity { id: 3ad43796-f8c3-44a0-813b-4ec411021df7, name: Name(Name("allow-ssh")), description: "allow inbound TCP connections on port 22 from anywhere", time_created: 2024-11-14T04:46:22.087449999Z, time_modified: 2024-11-14T04:46:22.087449999Z, time_deleted: None }, vpc_id: dea2096d-dbe9-4ffb-9b54-01f8106501ad, status: VpcFirewallRuleStatus(Enabled), direction: VpcFirewallRuleDirection(Inbound), targets: [VpcFirewallRuleTarget(Vpc(Name("default")))], filter_hosts: None, filter_ports: Some([L4PortRange(L4PortRange { first: L4Port(22), last: L4Port(22) })]), filter_protocols: Some([VpcFirewallRuleProtocol(Tcp)]), action: VpcFirewallRuleAction(Allow), priority: VpcFirewallRulePriority(VpcFirewallRulePriority(65534)) }, VpcFirewallRule { identity: VpcFirewallRuleIdentity { id: ac6740d8-c4f1-4168-bf9c-897964f2b5e3, name: Name(Name("allow-icmp")), description: "allow inbound ICMP traffic from anywhere", time_created: 2024-11-14T04:46:22.087450919Z, time_modified: 2024-11-14T04:46:22.087450919Z, time_deleted: None }, vpc_id: dea2096d-dbe9-4ffb-9b54-01f8106501ad, status: VpcFirewallRuleStatus(Enabled), direction: VpcFirewallRuleDirection(Inbound), targets: [VpcFirewallRuleTarget(Vpc(Name("default")))], filter_hosts: None, filter_ports: None, filter_protocols: Some([VpcFirewallRuleProtocol(Icmp)]), action: VpcFirewallRuleAction(Allow), priority: VpcFirewallRulePriority(VpcFirewallRulePriority(65534)) }]
26412024-11-14T04:46:22.087ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26422024-11-14T04:46:22.087ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26432024-11-14T04:46:22.089ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26442024-11-14T04:46:22.089ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26452024-11-14T04:46:22.092ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26462024-11-14T04:46:22.092ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26472024-11-14T04:46:22.094ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26482024-11-14T04:46:22.094ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26492024-11-14T04:46:22.096ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26502024-11-14T04:46:22.099ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26512024-11-14T04:46:22.099ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26522024-11-14T04:46:22.099ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcUpdateFirewall
26532024-11-14T04:46:22.151ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26542024-11-14T04:46:22.151ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Array [Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Array [Object {"type": String("vpc"), "value": String("default")}], "filter_ports": Null, "filter_protocols": Null, "identity": Object {"description": String("allow inbound traffic to all instances within the VPC if originated within the VPC"), "id": String("f867c751-a27a-4a13-8cca-8e2aaadbb702"), "name": String("allow-internal-inbound"), "time_created": String("2024-11-14T04:46:22.087447399Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087447399Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Array [String("22")], "filter_protocols": Array [String("TCP")], "identity": Object {"description": String("allow inbound TCP connections on port 22 from anywhere"), "id": String("3ad43796-f8c3-44a0-813b-4ec411021df7"), "name": String("allow-ssh"), "time_created": String("2024-11-14T04:46:22.087449999Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087449999Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Null, "filter_protocols": Array [String("ICMP")], "identity": Object {"description": String("allow inbound ICMP traffic from anywhere"), "id": String("ac6740d8-c4f1-4168-bf9c-897964f2b5e3"), "name": String("allow-icmp"), "time_created": String("2024-11-14T04:46:22.087450919Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087450919Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}]) node_id = 14 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26552024-11-14T04:46:22.154ZDEBGnexus (SEC): saga log event new_state = N015 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26562024-11-14T04:46:22.154ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 15 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26572024-11-14T04:46:22.158ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26582024-11-14T04:46:22.158ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26592024-11-14T04:46:22.161ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26602024-11-14T04:46:22.161ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26612024-11-14T04:46:22.163ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26622024-11-14T04:46:22.163ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26632024-11-14T04:46:22.166ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26642024-11-14T04:46:22.166ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26652024-11-14T04:46:22.168ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26662024-11-14T04:46:22.171ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26672024-11-14T04:46:22.171ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26682024-11-14T04:46:22.171ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26692024-11-14T04:46:22.174ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26702024-11-14T04:46:22.174ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26712024-11-14T04:46:22.177ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26722024-11-14T04:46:22.177ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26732024-11-14T04:46:22.180ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26742024-11-14T04:46:22.181ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26752024-11-14T04:46:22.185ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26762024-11-14T04:46:22.186ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26772024-11-14T04:46:22.187ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26782024-11-14T04:46:22.188ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26792024-11-14T04:46:22.190ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26802024-11-14T04:46:22.190ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26812024-11-14T04:46:22.192ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26822024-11-14T04:46:22.195ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") }, key: 6af791b4-74b7-406f-b55b-25de6525abe8, lookup_type: ById(6af791b4-74b7-406f-b55b-25de6525abe8) } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26832024-11-14T04:46:22.195ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26842024-11-14T04:46:22.196ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcCreateGateway
26852024-11-14T04:46:22.208ZDEBGnexus (SEC): saga log event new_state = N015 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26862024-11-14T04:46:22.208ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"key": String("6af791b4-74b7-406f-b55b-25de6525abe8"), "lookup_type": Object {"ById": String("6af791b4-74b7-406f-b55b-25de6525abe8")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}) node_id = 15 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26872024-11-14T04:46:22.212ZDEBGnexus (SEC): saga log event new_state = N016 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
26882024-11-14T04:46:22.212ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 16 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
26892024-11-14T04:46:22.215ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26902024-11-14T04:46:22.215ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26912024-11-14T04:46:22.217ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26922024-11-14T04:46:22.217ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26932024-11-14T04:46:22.219ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26942024-11-14T04:46:22.219ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26952024-11-14T04:46:22.221ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26962024-11-14T04:46:22.221ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26972024-11-14T04:46:22.223ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26982024-11-14T04:46:22.223ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
26992024-11-14T04:46:22.225ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27002024-11-14T04:46:22.225ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27012024-11-14T04:46:22.228ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27022024-11-14T04:46:22.228ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27032024-11-14T04:46:22.231ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27042024-11-14T04:46:22.233ZDEBGnexus (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ById(eabc1198-2a2f-4781-897a-051c378afa22) }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27052024-11-14T04:46:22.233ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27062024-11-14T04:46:22.234ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27072024-11-14T04:46:22.236ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27082024-11-14T04:46:22.236ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27092024-11-14T04:46:22.238ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27102024-11-14T04:46:22.239ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27112024-11-14T04:46:22.240ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27122024-11-14T04:46:22.240ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27132024-11-14T04:46:22.243ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27142024-11-14T04:46:22.246ZDEBGnexus (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ById(eabc1198-2a2f-4781-897a-051c378afa22) }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27152024-11-14T04:46:22.246ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27162024-11-14T04:46:22.246ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_node = VpcNotifySleds
27172024-11-14T04:46:22.251ZDEBGnexus: resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {} subnet_networks = {} vpc_interfaces = {}
27182024-11-14T04:46:22.251ZDEBGnexus: resolved firewall rules for sled agents sled_agent_rules = []
27192024-11-14T04:46:22.251ZDEBGnexus: resolved 0 rules for sleds
27202024-11-14T04:46:22.389ZDEBGnexus: resolved sleds for vpc default vpc_to_sled = []
27212024-11-14T04:46:22.389ZDEBGnexus: sending firewall rules to sled agents
27222024-11-14T04:46:22.389ZINFOnexus: updated firewall rules on 0 sleds file = nexus/networking/src/firewall_rules.rs:496
27232024-11-14T04:46:22.389ZDEBGnexus (SEC): saga log event new_state = N016 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27242024-11-14T04:46:22.389ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 16 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
27252024-11-14T04:46:22.393ZDEBGnexus (SEC): saga log event new_state = N017 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27262024-11-14T04:46:22.393ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 17 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
27272024-11-14T04:46:22.397ZDEBGnexus (SEC): saga log event new_state = N017 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27282024-11-14T04:46:22.397ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 17 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
27292024-11-14T04:46:22.401ZDEBGnexus (SEC): saga log event new_state = N019 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27302024-11-14T04:46:22.401ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 19 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
27312024-11-14T04:46:22.404ZDEBGnexus (SEC): saga log event new_state = N019 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27322024-11-14T04:46:22.404ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 19 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
27332024-11-14T04:46:22.407ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27342024-11-14T04:46:22.407ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
27352024-11-14T04:46:22.442ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27362024-11-14T04:46:22.443ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd saga_name = project-create saga_result = SagaResult { saga_id: 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd, saga_log: SagaLog { saga_id: 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd, unwinding: false, events: [N018 started, N018 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N013 started, N013 succeeded, N014 started, N014 succeeded, N015 started, N015 succeeded, N016 started, N016 succeeded, N017 started, N017 succeeded, N019 started, N019 succeeded], node_status: {0: Succeeded(Array [Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}, Object {"identity": Object {"description": String(""), "id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "name": String("proj-a32f5c8c37"), "time_created": String("2024-11-14T04:46:21.765323Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:21.765323Z")}, "rcgen": Number(1), "silo_id": String("992dae43-7740-49b5-814b-c5341916fc63")}]), 1: Succeeded(Object {"authz_project": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}, "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("992dae43-7740-49b5-814b-c5341916fc63"), "silo_user_id": String("7d466341-8999-4ad7-a241-ecff7d0e71de")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "vpc_create": Object {"description": String("Default VPC"), "dns_name": String("default"), "ipv6_prefix": String("fde3:3072:2af8::/48"), "name": String("default")}}), 2: Succeeded(Null), 3: Succeeded(String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")), 4: Succeeded(String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768")), 5: Succeeded(String("75dc1454-5a63-4592-b544-903cc9e95db0")), 6: Succeeded(String("3b514609-bf26-4103-b8a9-4a162e45f55a")), 7: Succeeded(String("64123868-fdd8-44b4-af58-803d800a258a")), 8: Succeeded(String("6af791b4-74b7-406f-b55b-25de6525abe8")), 9: Succeeded(Array [Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}, Object {"dns_name": String("default"), "firewall_gen": Number(1), "identity": Object {"description": String("Default VPC"), "id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "name": String("default"), "time_created": String("2024-11-14T04:46:21.879483Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:21.879483Z")}, "ipv6_prefix": String("fde3:3072:2af8::/48"), "project_id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "subnet_gen": Number(1), "system_router_id": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768"), "vni": Number(11383694)}]), 10: Succeeded(Object {"key": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768"), "lookup_type": Object {"ById": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Array [Object {"key": String("64123868-fdd8-44b4-af58-803d800a258a"), "lookup_type": Object {"ById": String("64123868-fdd8-44b4-af58-803d800a258a")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}, Object {"custom_router_id": Null, "identity": Object {"description": String("The default subnet for default"), "id": String("64123868-fdd8-44b4-af58-803d800a258a"), "name": String("default"), "time_created": String("2024-11-14T04:46:22.042084Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.042084Z")}, "ipv4_block": String("172.30.0.0/22"), "ipv6_block": String("fde3:3072:2af8::/64"), "rcgen": Number(1), "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}]), 14: Succeeded(Array [Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Array [Object {"type": String("vpc"), "value": String("default")}], "filter_ports": Null, "filter_protocols": Null, "identity": Object {"description": String("allow inbound traffic to all instances within the VPC if originated within the VPC"), "id": String("f867c751-a27a-4a13-8cca-8e2aaadbb702"), "name": String("allow-internal-inbound"), "time_created": String("2024-11-14T04:46:22.087447399Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087447399Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Array [String("22")], "filter_protocols": Array [String("TCP")], "identity": Object {"description": String("allow inbound TCP connections on port 22 from anywhere"), "id": String("3ad43796-f8c3-44a0-813b-4ec411021df7"), "name": String("allow-ssh"), "time_created": String("2024-11-14T04:46:22.087449999Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087449999Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Null, "filter_protocols": Array [String("ICMP")], "identity": Object {"description": String("allow inbound ICMP traffic from anywhere"), "id": String("ac6740d8-c4f1-4168-bf9c-897964f2b5e3"), "name": String("allow-icmp"), "time_created": String("2024-11-14T04:46:22.087450919Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087450919Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}]), 15: Succeeded(Object {"key": String("6af791b4-74b7-406f-b55b-25de6525abe8"), "lookup_type": Object {"ById": String("6af791b4-74b7-406f-b55b-25de6525abe8")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}), 16: Succeeded(Null), 17: Succeeded(Null), 18: Succeeded(Null), 19: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"default_internet_gateway_id": String("6af791b4-74b7-406f-b55b-25de6525abe8"), "default_subnet_id": String("64123868-fdd8-44b4-af58-803d800a258a"), "default_v4_route_id": String("75dc1454-5a63-4592-b544-903cc9e95db0"), "default_v6_route_id": String("3b514609-bf26-4103-b8a9-4a162e45f55a"), "firewall": Array [Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Array [Object {"type": String("vpc"), "value": String("default")}], "filter_ports": Null, "filter_protocols": Null, "identity": Object {"description": String("allow inbound traffic to all instances within the VPC if originated within the VPC"), "id": String("f867c751-a27a-4a13-8cca-8e2aaadbb702"), "name": String("allow-internal-inbound"), "time_created": String("2024-11-14T04:46:22.087447399Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087447399Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Array [String("22")], "filter_protocols": Array [String("TCP")], "identity": Object {"description": String("allow inbound TCP connections on port 22 from anywhere"), "id": String("3ad43796-f8c3-44a0-813b-4ec411021df7"), "name": String("allow-ssh"), "time_created": String("2024-11-14T04:46:22.087449999Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087449999Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Null, "filter_protocols": Array [String("ICMP")], "identity": Object {"description": String("allow inbound ICMP traffic from anywhere"), "id": String("ac6740d8-c4f1-4168-bf9c-897964f2b5e3"), "name": String("allow-icmp"), "time_created": String("2024-11-14T04:46:22.087450919Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.087450919Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}], "gateway": Object {"key": String("6af791b4-74b7-406f-b55b-25de6525abe8"), "lookup_type": Object {"ById": String("6af791b4-74b7-406f-b55b-25de6525abe8")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}, "no_result": Null, "project": Array [Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}, Object {"identity": Object {"description": String(""), "id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "name": String("proj-a32f5c8c37"), "time_created": String("2024-11-14T04:46:21.765323Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:21.765323Z")}, "rcgen": Number(1), "silo_id": String("992dae43-7740-49b5-814b-c5341916fc63")}], "route4": Null, "route6": Null, "router": Object {"key": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768"), "lookup_type": Object {"ById": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}, "subnet": Array [Object {"key": String("64123868-fdd8-44b4-af58-803d800a258a"), "lookup_type": Object {"ById": String("64123868-fdd8-44b4-af58-803d800a258a")}, "parent": Object {"key": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}}}, Object {"custom_router_id": Null, "identity": Object {"description": String("The default subnet for default"), "id": String("64123868-fdd8-44b4-af58-803d800a258a"), "name": String("default"), "time_created": String("2024-11-14T04:46:22.042084Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.042084Z")}, "ipv4_block": String("172.30.0.0/22"), "ipv6_block": String("fde3:3072:2af8::/64"), "rcgen": Number(1), "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")}], "system_router_id": String("fc9cb0b5-8ba0-4acb-8aa7-669d702de768"), "vpc": Null, "vpc_create_params": Object {"authz_project": Object {"key": String("eabc1198-2a2f-4781-897a-051c378afa22"), "lookup_type": Object {"ByName": String("proj-a32f5c8c37")}, "parent": Object {"key": String("992dae43-7740-49b5-814b-c5341916fc63"), "lookup_type": Object {"ById": String("992dae43-7740-49b5-814b-c5341916fc63")}, "parent": Null}}, "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("992dae43-7740-49b5-814b-c5341916fc63"), "silo_user_id": String("7d466341-8999-4ad7-a241-ecff7d0e71de")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "vpc_create": Object {"description": String("Default VPC"), "dns_name": String("default"), "ipv6_prefix": String("fde3:3072:2af8::/48"), "name": String("default")}}, "vpc_id": String("dea2096d-dbe9-4ffb-9b54-01f8106501ad")} }) }
27372024-11-14T04:46:22.443ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 789983 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 548170c2-0ff4-47aa-a118-6f031f0801ab response_code = 201 uri = /v1/projects
27382024-11-14T04:46:22.447ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27392024-11-14T04:46:22.448ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27402024-11-14T04:46:22.449ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27412024-11-14T04:46:22.449ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27422024-11-14T04:46:22.451ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
27432024-11-14T04:46:22.452ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
27442024-11-14T04:46:22.452ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27452024-11-14T04:46:22.452ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27462024-11-14T04:46:22.454ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27472024-11-14T04:46:22.454ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27482024-11-14T04:46:22.456ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27492024-11-14T04:46:22.456ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27502024-11-14T04:46:22.457ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27512024-11-14T04:46:22.458ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27522024-11-14T04:46:22.459ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
27532024-11-14T04:46:22.460ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
27542024-11-14T04:46:22.460ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27552024-11-14T04:46:22.461ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27562024-11-14T04:46:22.462ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
27572024-11-14T04:46:22.463ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
27582024-11-14T04:46:22.463ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27592024-11-14T04:46:22.463ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27602024-11-14T04:46:22.467ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27612024-11-14T04:46:22.467ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27622024-11-14T04:46:22.468ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27632024-11-14T04:46:22.468ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27642024-11-14T04:46:22.469ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27652024-11-14T04:46:22.469ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27662024-11-14T04:46:22.471ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27672024-11-14T04:46:22.471ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27682024-11-14T04:46:22.472ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
27692024-11-14T04:46:22.473ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
27702024-11-14T04:46:22.474ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27712024-11-14T04:46:22.474ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27722024-11-14T04:46:22.475ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27732024-11-14T04:46:22.475ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27742024-11-14T04:46:22.477ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
27752024-11-14T04:46:22.477ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
27762024-11-14T04:46:22.478ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
27772024-11-14T04:46:22.479ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
27782024-11-14T04:46:22.479ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f roles = RoleSet { roles: {} } uri = /v1/disks?project=proj-a32f5c8c37
27792024-11-14T04:46:22.479ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f resource = Database result = Ok(()) uri = /v1/disks?project=proj-a32f5c8c37
27802024-11-14T04:46:22.481ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f roles = RoleSet { roles: {} } uri = /v1/disks?project=proj-a32f5c8c37
27812024-11-14T04:46:22.481ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f resource = Database result = Ok(()) uri = /v1/disks?project=proj-a32f5c8c37
27822024-11-14T04:46:22.483ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f roles = RoleSet { roles: {} } uri = /v1/disks?project=proj-a32f5c8c37
27832024-11-14T04:46:22.483ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f resource = Database result = Ok(()) uri = /v1/disks?project=proj-a32f5c8c37
27842024-11-14T04:46:22.486ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f roles = RoleSet { roles: {} } uri = /v1/disks?project=proj-a32f5c8c37
27852024-11-14T04:46:22.486ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f resource = Database result = Ok(()) uri = /v1/disks?project=proj-a32f5c8c37
27862024-11-14T04:46:22.488ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f roles = RoleSet { roles: {} } uri = /v1/disks?project=proj-a32f5c8c37
27872024-11-14T04:46:22.488ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f resource = Database result = Ok(()) uri = /v1/disks?project=proj-a32f5c8c37
27882024-11-14T04:46:22.490ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f roles = RoleSet { roles: {} } uri = /v1/disks?project=proj-a32f5c8c37
27892024-11-14T04:46:22.490ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f resource = Database result = Ok(()) uri = /v1/disks?project=proj-a32f5c8c37
27902024-11-14T04:46:22.493ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/disks?project=proj-a32f5c8c37
27912024-11-14T04:46:22.495ZDEBGnexus (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f resource = Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") } result = Ok(()) uri = /v1/disks?project=proj-a32f5c8c37
27922024-11-14T04:46:22.495ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create
27932024-11-14T04:46:22.495ZINFOnexus (SEC): saga create dag = {"end_node":10,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,0,null],[8,10,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateDiskId","name":"disk_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateVolumeId","name":"volume_id"}},{"Action":{"action_name":"disk_create.create_disk_record","label":"CreateDiskRecord","name":"created_disk"}},{"Action":{"action_name":"disk_create.regions_alloc","label":"RegionsAlloc","name":"datasets_and_regions"}},{"Action":{"action_name":"disk_create.space_account","label":"SpaceAccount","name":"no_result"}},{"Action":{"action_name":"disk_create.regions_ensure_undo","label":"RegionsEnsureUndo","name":"regions_ensure_undo"}},{"Action":{"action_name":"disk_create.regions_ensure","label":"RegionsEnsure","name":"regions_ensure"}},{"Action":{"action_name":"disk_create.create_volume_record","label":"CreateVolumeRecord","name":"created_volume"}},{"Action":{"action_name":"disk_create.finalize_disk_record","label":"FinalizeDiskRecord","name":"disk_runtime"}},{"Start":{"params":{"create_params":{"description":"","disk_source":{"block_size":512,"type":"blank"},"name":"disk-55f94a4ba62","size":1073741824},"project_id":"eabc1198-2a2f-4781-897a-051c378afa22","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","silo_user_id":"7d466341-8999-4ad7-a241-ecff7d0e71de"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"disk-create","start_node":9} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27942024-11-14T04:46:22.495ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create
27952024-11-14T04:46:22.498ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create
27962024-11-14T04:46:22.498ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27972024-11-14T04:46:22.498ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
27982024-11-14T04:46:22.498ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
27992024-11-14T04:46:22.502ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28002024-11-14T04:46:22.502ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28012024-11-14T04:46:22.505ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28022024-11-14T04:46:22.505ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28032024-11-14T04:46:22.508ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28042024-11-14T04:46:22.508ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("f4ffd013-77da-4445-90b9-b231fcd65cef")) node_id = 0 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28052024-11-14T04:46:22.511ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28062024-11-14T04:46:22.511ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28072024-11-14T04:46:22.515ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28082024-11-14T04:46:22.515ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")) node_id = 1 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28092024-11-14T04:46:22.518ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28102024-11-14T04:46:22.518ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28112024-11-14T04:46:22.521ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28122024-11-14T04:46:22.522ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28132024-11-14T04:46:22.523ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28142024-11-14T04:46:22.523ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28152024-11-14T04:46:22.524ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28162024-11-14T04:46:22.524ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28172024-11-14T04:46:22.526ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28182024-11-14T04:46:22.527ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28192024-11-14T04:46:22.530ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28202024-11-14T04:46:22.530ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28212024-11-14T04:46:22.533ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28222024-11-14T04:46:22.533ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28232024-11-14T04:46:22.536ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28242024-11-14T04:46:22.539ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ById(eabc1198-2a2f-4781-897a-051c378afa22) } result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28252024-11-14T04:46:22.539ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28262024-11-14T04:46:22.539ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28272024-11-14T04:46:22.541ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28282024-11-14T04:46:22.541ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28292024-11-14T04:46:22.544ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28302024-11-14T04:46:22.544ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28312024-11-14T04:46:22.547ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28322024-11-14T04:46:22.547ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28332024-11-14T04:46:22.549ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28342024-11-14T04:46:22.551ZDEBGnexus (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ById(eabc1198-2a2f-4781-897a-051c378afa22) } result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28352024-11-14T04:46:22.551ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28362024-11-14T04:46:22.551ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = CreateDiskRecord
28372024-11-14T04:46:22.586ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28382024-11-14T04:46:22.586ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"block_size": String("Traditional"), "create_image_id": Null, "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("f4ffd013-77da-4445-90b9-b231fcd65cef"), "name": String("disk-55f94a4ba62"), "time_created": String("2024-11-14T04:46:22.521820Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.521820Z")}, "pantry_address": Null, "project_id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2024-11-14T04:46:22.521820Z")}, "size": Number(1073741824), "slot": Null, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}) node_id = 2 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28392024-11-14T04:46:22.594ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28402024-11-14T04:46:22.594ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28412024-11-14T04:46:22.597ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = RegionsAlloc
28422024-11-14T04:46:22.597ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = RegionsAlloc
28432024-11-14T04:46:22.679ZINFOnexus (datastore): Allocated regions for volume datasets_and_regions = [(Dataset { identity: DatasetIdentity { id: 212c8858-971f-4450-8be6-797b7f642d45, time_created: 2024-11-14T04:46:16.023642Z, time_modified: 2024-11-14T04:46:16.023642Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: d5ec0ee3-bbc0-4fc4-b92f-6c8fc22d4e7b, ip: Some(fd00:1122:3344:101::17), port: Some(SqlU16(32345)), kind: Crucible, size_used: Some(1073741824), zone_name: None, quota: None, reservation: None, compression: None }, Region { identity: RegionIdentity { id: eb182fec-aaab-45c0-bcc0-f443ebe55c2c, time_created: 2024-11-14T04:46:22.638157Z, time_modified: 2024-11-14T04:46:22.638157Z }, dataset_id: 212c8858-971f-4450-8be6-797b7f642d45, volume_id: 77777d5b-f8f3-4f3c-8c81-75fcbce5d832, block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false }), (Dataset { identity: DatasetIdentity { id: 9f36a31f-42f0-46df-8818-c53c148060a5, time_created: 2024-11-14T04:46:16.023643Z, time_modified: 2024-11-14T04:46:16.023643Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: d0e3c514-0a4e-463a-96d3-0dd3fac40f52, ip: Some(fd00:1122:3344:101::14), port: Some(SqlU16(32345)), kind: Crucible, size_used: Some(1073741824), zone_name: None, quota: None, reservation: None, compression: None }, Region { identity: RegionIdentity { id: b94f4172-ff19-4b0a-931a-7eab974838b6, time_created: 2024-11-14T04:46:22.638157Z, time_modified: 2024-11-14T04:46:22.638157Z }, dataset_id: 9f36a31f-42f0-46df-8818-c53c148060a5, volume_id: 77777d5b-f8f3-4f3c-8c81-75fcbce5d832, block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false }), (Dataset { identity: DatasetIdentity { id: d11873f3-0c17-405b-a55d-f221a6ae1b37, time_created: 2024-11-14T04:46:16.023642Z, time_modified: 2024-11-14T04:46:16.023642Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 42743ca1-d71f-4721-aaf3-5ac7d5df758b, ip: Some(fd00:1122:3344:101::15), port: Some(SqlU16(32345)), kind: Crucible, size_used: Some(1073741824), zone_name: None, quota: None, reservation: None, compression: None }, Region { identity: RegionIdentity { id: 5e6ec445-eaf0-41a7-b58c-e17e36cb0697, time_created: 2024-11-14T04:46:22.638157Z, time_modified: 2024-11-14T04:46:22.638157Z }, dataset_id: d11873f3-0c17-405b-a55d-f221a6ae1b37, volume_id: 77777d5b-f8f3-4f3c-8c81-75fcbce5d832, block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false })] file = nexus/db-queries/src/db/datastore/region.rs:280 maybe_snapshot_id = None volume_id = 77777d5b-f8f3-4f3c-8c81-75fcbce5d832
28442024-11-14T04:46:22.679ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28452024-11-14T04:46:22.679ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Array [Array [Object {"compression": Null, "identity": Object {"id": String("212c8858-971f-4450-8be6-797b7f642d45"), "time_created": String("2024-11-14T04:46:16.023642Z"), "time_modified": String("2024-11-14T04:46:16.023642Z")}, "ip": String("fd00:1122:3344:101::17"), "kind": String("Crucible"), "pool_id": String("d5ec0ee3-bbc0-4fc4-b92f-6c8fc22d4e7b"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("212c8858-971f-4450-8be6-797b7f642d45"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb182fec-aaab-45c0-bcc0-f443ebe55c2c"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}], Array [Object {"compression": Null, "identity": Object {"id": String("9f36a31f-42f0-46df-8818-c53c148060a5"), "time_created": String("2024-11-14T04:46:16.023643Z"), "time_modified": String("2024-11-14T04:46:16.023643Z")}, "ip": String("fd00:1122:3344:101::14"), "kind": String("Crucible"), "pool_id": String("d0e3c514-0a4e-463a-96d3-0dd3fac40f52"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9f36a31f-42f0-46df-8818-c53c148060a5"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("b94f4172-ff19-4b0a-931a-7eab974838b6"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}], Array [Object {"compression": Null, "identity": Object {"id": String("d11873f3-0c17-405b-a55d-f221a6ae1b37"), "time_created": String("2024-11-14T04:46:16.023642Z"), "time_modified": String("2024-11-14T04:46:16.023642Z")}, "ip": String("fd00:1122:3344:101::15"), "kind": String("Crucible"), "pool_id": String("42743ca1-d71f-4721-aaf3-5ac7d5df758b"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("d11873f3-0c17-405b-a55d-f221a6ae1b37"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("5e6ec445-eaf0-41a7-b58c-e17e36cb0697"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}]]) node_id = 3 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28462024-11-14T04:46:22.688ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28472024-11-14T04:46:22.688ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28482024-11-14T04:46:22.692ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = SpaceAccount
28492024-11-14T04:46:22.692ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = SpaceAccount
28502024-11-14T04:46:22.715ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28512024-11-14T04:46:22.715ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28522024-11-14T04:46:22.719ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28532024-11-14T04:46:22.719ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28542024-11-14T04:46:22.722ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28552024-11-14T04:46:22.722ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28562024-11-14T04:46:22.725ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28572024-11-14T04:46:22.725ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28582024-11-14T04:46:22.739ZWARNnexus (SagaExecutor): Region requested, not yet created. Retrying in 143.803389ms dataset = 9f36a31f-42f0-46df-8818-c53c148060a5 file = nexus/src/app/crucible.rs:230 region = b94f4172-ff19-4b0a-931a-7eab974838b6 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create
28592024-11-14T04:46:22.739ZWARNnexus (SagaExecutor): Region requested, not yet created. Retrying in 220.361697ms dataset = 212c8858-971f-4450-8be6-797b7f642d45 file = nexus/src/app/crucible.rs:230 region = eb182fec-aaab-45c0-bcc0-f443ebe55c2c saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create
28602024-11-14T04:46:22.747ZWARNnexus (SagaExecutor): Region requested, not yet created. Retrying in 251.095643ms dataset = d11873f3-0c17-405b-a55d-f221a6ae1b37 file = nexus/src/app/crucible.rs:230 region = 5e6ec445-eaf0-41a7-b58c-e17e36cb0697 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create
28612024-11-14T04:46:22.891ZWARNnexus (SagaExecutor): Region requested, not yet created. Retrying in 535.107441ms dataset = 9f36a31f-42f0-46df-8818-c53c148060a5 file = nexus/src/app/crucible.rs:230 region = b94f4172-ff19-4b0a-931a-7eab974838b6 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create
28622024-11-14T04:46:22.964ZWARNnexus (SagaExecutor): Region requested, not yet created. Retrying in 342.181109ms dataset = 212c8858-971f-4450-8be6-797b7f642d45 file = nexus/src/app/crucible.rs:230 region = eb182fec-aaab-45c0-bcc0-f443ebe55c2c saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create
28632024-11-14T04:46:23.442ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28642024-11-14T04:46:23.442ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("{\\"type\\":\\"volume\\",\\"block_size\\":512,\\"id\\":\\"f4ffd013-77da-4445-90b9-b231fcd65cef\\",\\"sub_volumes\\":[{\\"type\\":\\"region\\",\\"block_size\\":512,\\"blocks_per_extent\\":131072,\\"extent_count\\":16,\\"gen\\":1,\\"opts\\":{\\"id\\":\\"f4ffd013-77da-4445-90b9-b231fcd65cef\\",\\"key\\":\\"vCsDHy45QKHVne9TMlv3wrJk/aAq3V7PiNANG9TRwhc=\\",\\"lossy\\":false,\\"read_only\\":false,\\"target\\":[\\"[fd00:1122:3344:101::15]:19000\\",\\"[fd00:1122:3344:101::17]:19000\\",\\"[fd00:1122:3344:101::14]:19000\\"]}}]}")) node_id = 6 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28652024-11-14T04:46:23.446ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28662024-11-14T04:46:23.446ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28672024-11-14T04:46:23.473ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28682024-11-14T04:46:23.473ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 7 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28692024-11-14T04:46:23.476ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
28702024-11-14T04:46:23.476ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
28712024-11-14T04:46:23.480ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28722024-11-14T04:46:23.480ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28732024-11-14T04:46:23.483ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28742024-11-14T04:46:23.483ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28752024-11-14T04:46:23.484ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28762024-11-14T04:46:23.485ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28772024-11-14T04:46:23.486ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28782024-11-14T04:46:23.486ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28792024-11-14T04:46:23.488ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28802024-11-14T04:46:23.488ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28812024-11-14T04:46:23.490ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28822024-11-14T04:46:23.491ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28832024-11-14T04:46:23.493ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28842024-11-14T04:46:23.493ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28852024-11-14T04:46:23.496ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28862024-11-14T04:46:23.499ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Disk { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ById(eabc1198-2a2f-4781-897a-051c378afa22) }, key: f4ffd013-77da-4445-90b9-b231fcd65cef, lookup_type: ById(f4ffd013-77da-4445-90b9-b231fcd65cef) } result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28872024-11-14T04:46:23.499ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28882024-11-14T04:46:23.499ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28892024-11-14T04:46:23.501ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28902024-11-14T04:46:23.501ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28912024-11-14T04:46:23.503ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28922024-11-14T04:46:23.503ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28932024-11-14T04:46:23.505ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28942024-11-14T04:46:23.505ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28952024-11-14T04:46:23.507ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28962024-11-14T04:46:23.510ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Disk { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ById(eabc1198-2a2f-4781-897a-051c378afa22) }, key: f4ffd013-77da-4445-90b9-b231fcd65cef, lookup_type: ById(f4ffd013-77da-4445-90b9-b231fcd65cef) } result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28972024-11-14T04:46:23.510ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28982024-11-14T04:46:23.510ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_node = FinalizeDiskRecord
28992024-11-14T04:46:23.523ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29002024-11-14T04:46:23.523ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
29012024-11-14T04:46:23.527ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29022024-11-14T04:46:23.527ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
29032024-11-14T04:46:23.530ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29042024-11-14T04:46:23.530ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
29052024-11-14T04:46:23.533ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29062024-11-14T04:46:23.533ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
29072024-11-14T04:46:23.548ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29082024-11-14T04:46:23.549ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7 saga_name = disk-create saga_result = SagaResult { saga_id: fd45f922-2194-4e21-821b-92d56fca70f7, saga_log: SagaLog { saga_id: fd45f922-2194-4e21-821b-92d56fca70f7, unwinding: false, events: [N009 started, N009 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N010 started, N010 succeeded], node_status: {0: Succeeded(String("f4ffd013-77da-4445-90b9-b231fcd65cef")), 1: Succeeded(String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")), 2: Succeeded(Object {"block_size": String("Traditional"), "create_image_id": Null, "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("f4ffd013-77da-4445-90b9-b231fcd65cef"), "name": String("disk-55f94a4ba62"), "time_created": String("2024-11-14T04:46:22.521820Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.521820Z")}, "pantry_address": Null, "project_id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2024-11-14T04:46:22.521820Z")}, "size": Number(1073741824), "slot": Null, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}), 3: Succeeded(Array [Array [Object {"compression": Null, "identity": Object {"id": String("212c8858-971f-4450-8be6-797b7f642d45"), "time_created": String("2024-11-14T04:46:16.023642Z"), "time_modified": String("2024-11-14T04:46:16.023642Z")}, "ip": String("fd00:1122:3344:101::17"), "kind": String("Crucible"), "pool_id": String("d5ec0ee3-bbc0-4fc4-b92f-6c8fc22d4e7b"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("212c8858-971f-4450-8be6-797b7f642d45"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb182fec-aaab-45c0-bcc0-f443ebe55c2c"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}], Array [Object {"compression": Null, "identity": Object {"id": String("9f36a31f-42f0-46df-8818-c53c148060a5"), "time_created": String("2024-11-14T04:46:16.023643Z"), "time_modified": String("2024-11-14T04:46:16.023643Z")}, "ip": String("fd00:1122:3344:101::14"), "kind": String("Crucible"), "pool_id": String("d0e3c514-0a4e-463a-96d3-0dd3fac40f52"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9f36a31f-42f0-46df-8818-c53c148060a5"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("b94f4172-ff19-4b0a-931a-7eab974838b6"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}], Array [Object {"compression": Null, "identity": Object {"id": String("d11873f3-0c17-405b-a55d-f221a6ae1b37"), "time_created": String("2024-11-14T04:46:16.023642Z"), "time_modified": String("2024-11-14T04:46:16.023642Z")}, "ip": String("fd00:1122:3344:101::15"), "kind": String("Crucible"), "pool_id": String("42743ca1-d71f-4721-aaf3-5ac7d5df758b"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("d11873f3-0c17-405b-a55d-f221a6ae1b37"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("5e6ec445-eaf0-41a7-b58c-e17e36cb0697"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}]]), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(String("{\\"type\\":\\"volume\\",\\"block_size\\":512,\\"id\\":\\"f4ffd013-77da-4445-90b9-b231fcd65cef\\",\\"sub_volumes\\":[{\\"type\\":\\"region\\",\\"block_size\\":512,\\"blocks_per_extent\\":131072,\\"extent_count\\":16,\\"gen\\":1,\\"opts\\":{\\"id\\":\\"f4ffd013-77da-4445-90b9-b231fcd65cef\\",\\"key\\":\\"vCsDHy45QKHVne9TMlv3wrJk/aAq3V7PiNANG9TRwhc=\\",\\"lossy\\":false,\\"read_only\\":false,\\"target\\":[\\"[fd00:1122:3344:101::15]:19000\\",\\"[fd00:1122:3344:101::17]:19000\\",\\"[fd00:1122:3344:101::14]:19000\\"]}}]}")), 7: Succeeded(Null), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"created_disk": Object {"block_size": String("Traditional"), "create_image_id": Null, "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("f4ffd013-77da-4445-90b9-b231fcd65cef"), "name": String("disk-55f94a4ba62"), "time_created": String("2024-11-14T04:46:22.521820Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.521820Z")}, "pantry_address": Null, "project_id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2024-11-14T04:46:22.521820Z")}, "size": Number(1073741824), "slot": Null, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}, "created_volume": Null, "datasets_and_regions": Array [Array [Object {"compression": Null, "identity": Object {"id": String("212c8858-971f-4450-8be6-797b7f642d45"), "time_created": String("2024-11-14T04:46:16.023642Z"), "time_modified": String("2024-11-14T04:46:16.023642Z")}, "ip": String("fd00:1122:3344:101::17"), "kind": String("Crucible"), "pool_id": String("d5ec0ee3-bbc0-4fc4-b92f-6c8fc22d4e7b"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("212c8858-971f-4450-8be6-797b7f642d45"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb182fec-aaab-45c0-bcc0-f443ebe55c2c"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}], Array [Object {"compression": Null, "identity": Object {"id": String("9f36a31f-42f0-46df-8818-c53c148060a5"), "time_created": String("2024-11-14T04:46:16.023643Z"), "time_modified": String("2024-11-14T04:46:16.023643Z")}, "ip": String("fd00:1122:3344:101::14"), "kind": String("Crucible"), "pool_id": String("d0e3c514-0a4e-463a-96d3-0dd3fac40f52"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9f36a31f-42f0-46df-8818-c53c148060a5"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("b94f4172-ff19-4b0a-931a-7eab974838b6"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}], Array [Object {"compression": Null, "identity": Object {"id": String("d11873f3-0c17-405b-a55d-f221a6ae1b37"), "time_created": String("2024-11-14T04:46:16.023642Z"), "time_modified": String("2024-11-14T04:46:16.023642Z")}, "ip": String("fd00:1122:3344:101::15"), "kind": String("Crucible"), "pool_id": String("42743ca1-d71f-4721-aaf3-5ac7d5df758b"), "port": Number(32345), "quota": Null, "rcgen": Number(1), "reservation": Null, "size_used": Number(1073741824), "time_deleted": Null, "zone_name": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("d11873f3-0c17-405b-a55d-f221a6ae1b37"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("5e6ec445-eaf0-41a7-b58c-e17e36cb0697"), "time_created": String("2024-11-14T04:46:22.638157Z"), "time_modified": String("2024-11-14T04:46:22.638157Z")}, "port": Null, "read_only": Bool(false), "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}]], "disk_id": String("f4ffd013-77da-4445-90b9-b231fcd65cef"), "disk_runtime": Null, "no_result": Null, "regions_ensure": String("{\\"type\\":\\"volume\\",\\"block_size\\":512,\\"id\\":\\"f4ffd013-77da-4445-90b9-b231fcd65cef\\",\\"sub_volumes\\":[{\\"type\\":\\"region\\",\\"block_size\\":512,\\"blocks_per_extent\\":131072,\\"extent_count\\":16,\\"gen\\":1,\\"opts\\":{\\"id\\":\\"f4ffd013-77da-4445-90b9-b231fcd65cef\\",\\"key\\":\\"vCsDHy45QKHVne9TMlv3wrJk/aAq3V7PiNANG9TRwhc=\\",\\"lossy\\":false,\\"read_only\\":false,\\"target\\":[\\"[fd00:1122:3344:101::15]:19000\\",\\"[fd00:1122:3344:101::17]:19000\\",\\"[fd00:1122:3344:101::14]:19000\\"]}}]}"), "regions_ensure_undo": Null, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")} }) }
29092024-11-14T04:46:23.549ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1102549 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 688bd720-7eca-4125-8625-34ca0f79385f response_code = 201 uri = /v1/disks?project=proj-a32f5c8c37
29102024-11-14T04:46:23.550ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29112024-11-14T04:46:23.551ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29122024-11-14T04:46:23.552ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29132024-11-14T04:46:23.552ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29142024-11-14T04:46:23.553ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
29152024-11-14T04:46:23.555ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
29162024-11-14T04:46:23.555ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29172024-11-14T04:46:23.555ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29182024-11-14T04:46:23.556ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29192024-11-14T04:46:23.556ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29202024-11-14T04:46:23.557ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29212024-11-14T04:46:23.557ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29222024-11-14T04:46:23.559ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29232024-11-14T04:46:23.559ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29242024-11-14T04:46:23.561ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
29252024-11-14T04:46:23.562ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
29262024-11-14T04:46:23.562ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29272024-11-14T04:46:23.562ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29282024-11-14T04:46:23.563ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
29292024-11-14T04:46:23.565ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
29302024-11-14T04:46:23.565ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29312024-11-14T04:46:23.565ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29322024-11-14T04:46:23.567ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29332024-11-14T04:46:23.567ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29342024-11-14T04:46:23.569ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29352024-11-14T04:46:23.569ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29362024-11-14T04:46:23.570ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29372024-11-14T04:46:23.570ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29382024-11-14T04:46:23.572ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29392024-11-14T04:46:23.573ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29402024-11-14T04:46:23.574ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
29412024-11-14T04:46:23.575ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
29422024-11-14T04:46:23.575ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29432024-11-14T04:46:23.575ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29442024-11-14T04:46:23.576ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29452024-11-14T04:46:23.576ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29462024-11-14T04:46:23.578ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
29472024-11-14T04:46:23.578ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
29482024-11-14T04:46:23.579ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
29492024-11-14T04:46:23.580ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
29502024-11-14T04:46:23.580ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29512024-11-14T04:46:23.581ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29522024-11-14T04:46:23.582ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29532024-11-14T04:46:23.582ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29542024-11-14T04:46:23.584ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29552024-11-14T04:46:23.585ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29562024-11-14T04:46:23.586ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29572024-11-14T04:46:23.587ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29582024-11-14T04:46:23.589ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29592024-11-14T04:46:23.589ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29602024-11-14T04:46:23.591ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29612024-11-14T04:46:23.591ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29622024-11-14T04:46:23.594ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29632024-11-14T04:46:23.594ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29642024-11-14T04:46:23.596ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29652024-11-14T04:46:23.599ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Disk { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: f4ffd013-77da-4445-90b9-b231fcd65cef, lookup_type: ByName("disk-55f94a4ba62") } result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29662024-11-14T04:46:23.599ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29672024-11-14T04:46:23.599ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29682024-11-14T04:46:23.601ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29692024-11-14T04:46:23.601ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29702024-11-14T04:46:23.602ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29712024-11-14T04:46:23.602ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29722024-11-14T04:46:23.604ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29732024-11-14T04:46:23.604ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29742024-11-14T04:46:23.606ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29752024-11-14T04:46:23.606ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29762024-11-14T04:46:23.609ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29772024-11-14T04:46:23.609ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29782024-11-14T04:46:23.610ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29792024-11-14T04:46:23.611ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Database result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29802024-11-14T04:46:23.613ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29812024-11-14T04:46:23.618ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e resource = Disk { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ById(eabc1198-2a2f-4781-897a-051c378afa22) }, key: f4ffd013-77da-4445-90b9-b231fcd65cef, lookup_type: ById(f4ffd013-77da-4445-90b9-b231fcd65cef) } result = Ok(()) uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
29822024-11-14T04:46:23.618ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
29832024-11-14T04:46:23.618ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,5,null],[4,6,null],[6,7,null],[5,7,null],[7,8,null],[8,9,null],[9,10,null],[10,11,null],[3,4,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"disk_delete.delete_disk_record","label":"DeleteDiskRecord","name":"deleted_disk"}},{"Action":{"action_name":"disk_delete.space_account","label":"SpaceAccount","name":"no_result1"}},{"Constant":{"name":"params_for_volume_delete_subsaga","value":{"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","silo_user_id":"7d466341-8999-4ad7-a241-ecff7d0e71de"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}},"volume_id":"77777d5b-f8f3-4f3c-8c81-75fcbce5d832"}}},{"SubsagaStart":{"params_node_name":"params_for_volume_delete_subsaga","saga_name":"volume-delete"}},{"Action":{"action_name":"volume_delete.decrease_crucible_resource_count","label":"DecreaseCrucibleResourceCount","name":"crucible_resources_to_delete"}},{"Action":{"action_name":"volume_delete.delete_crucible_regions","label":"DeleteCrucibleRegions","name":"no_result_1"}},{"Action":{"action_name":"volume_delete.delete_crucible_running_snapshots","label":"DeleteCrucibleRunningSnapshots","name":"no_result_2"}},{"Action":{"action_name":"volume_delete.delete_crucible_snapshots","label":"DeleteCrucibleSnapshots","name":"no_result_3"}},{"Action":{"action_name":"volume_delete.delete_crucible_snapshot_records","label":"DeleteCrucibleSnapshotRecords","name":"no_result_4"}},{"Action":{"action_name":"volume_delete.find_freed_crucible_regions","label":"FindFreedCrucibleRegions","name":"freed_crucible_regions"}},{"Action":{"action_name":"volume_delete.delete_freed_crucible_regions","label":"DeleteFreedCrucibleRegions","name":"no_result_5"}},{"Action":{"action_name":"volume_delete.hard_delete_volume_record","label":"HardDeleteVolumeRecord","name":"volume_hard_deleted"}},{"SubsagaEnd":{"name":"volume_delete_subsaga_no_result"}},{"Start":{"params":{"disk_id":"f4ffd013-77da-4445-90b9-b231fcd65cef","project_id":"eabc1198-2a2f-4781-897a-051c378afa22","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","silo_user_id":"7d466341-8999-4ad7-a241-ecff7d0e71de"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}},"volume_id":"77777d5b-f8f3-4f3c-8c81-75fcbce5d832"}}},"End"]},"saga_name":"disk-delete","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29842024-11-14T04:46:23.618ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
29852024-11-14T04:46:23.623ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
29862024-11-14T04:46:23.623ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29872024-11-14T04:46:23.623ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29882024-11-14T04:46:23.623ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
29892024-11-14T04:46:23.637ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29902024-11-14T04:46:23.637ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
29912024-11-14T04:46:23.642ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29922024-11-14T04:46:23.642ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
29932024-11-14T04:46:23.664ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29942024-11-14T04:46:23.664ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"block_size": String("Traditional"), "create_image_id": Null, "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("f4ffd013-77da-4445-90b9-b231fcd65cef"), "name": String("disk-55f94a4ba62"), "time_created": String("2024-11-14T04:46:22.521820Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.521820Z")}, "pantry_address": Null, "project_id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2024-11-14T04:46:23.499072Z")}, "size": Number(1073741824), "slot": Null, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}) node_id = 0 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
29952024-11-14T04:46:23.667ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
29962024-11-14T04:46:23.667ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
29972024-11-14T04:46:23.672ZDEBGnexus (SagaExecutor): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true roles = RoleSet { roles: {} } saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete saga_node = SpaceAccount
29982024-11-14T04:46:23.672ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true resource = Database result = Ok(()) saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete saga_node = SpaceAccount
29992024-11-14T04:46:23.700ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30002024-11-14T04:46:23.700ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 1 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30012024-11-14T04:46:23.703ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30022024-11-14T04:46:23.703ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30032024-11-14T04:46:23.707ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30042024-11-14T04:46:23.707ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("992dae43-7740-49b5-814b-c5341916fc63"), "silo_user_id": String("7d466341-8999-4ad7-a241-ecff7d0e71de")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}) node_id = 2 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30052024-11-14T04:46:23.710ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30062024-11-14T04:46:23.710ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30072024-11-14T04:46:23.713ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30082024-11-14T04:46:23.713ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30092024-11-14T04:46:23.716ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30102024-11-14T04:46:23.716ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30112024-11-14T04:46:23.742ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30122024-11-14T04:46:23.742ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("5e6ec445-eaf0-41a7-b58c-e17e36cb0697"), String("eb182fec-aaab-45c0-bcc0-f443ebe55c2c"), String("b94f4172-ff19-4b0a-931a-7eab974838b6")]}}) node_id = 4 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30132024-11-14T04:46:23.746ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30142024-11-14T04:46:23.746ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30152024-11-14T04:46:23.749ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30162024-11-14T04:46:23.749ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30172024-11-14T04:46:23.752ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30182024-11-14T04:46:23.752ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30192024-11-14T04:46:23.784ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 141.327952ms file = nexus/src/app/crucible.rs:827 region_id = eb182fec-aaab-45c0-bcc0-f443ebe55c2c saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
30202024-11-14T04:46:23.784ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 144.506157ms file = nexus/src/app/crucible.rs:827 region_id = 5e6ec445-eaf0-41a7-b58c-e17e36cb0697 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
30212024-11-14T04:46:23.785ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 144.585557ms file = nexus/src/app/crucible.rs:827 region_id = b94f4172-ff19-4b0a-931a-7eab974838b6 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
30222024-11-14T04:46:23.930ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:812 region_id = eb182fec-aaab-45c0-bcc0-f443ebe55c2c saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
30232024-11-14T04:46:23.933ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:812 region_id = 5e6ec445-eaf0-41a7-b58c-e17e36cb0697 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
30242024-11-14T04:46:23.934ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:812 region_id = b94f4172-ff19-4b0a-931a-7eab974838b6 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
30252024-11-14T04:46:23.950ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30262024-11-14T04:46:23.950ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30272024-11-14T04:46:23.954ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30282024-11-14T04:46:23.954ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30292024-11-14T04:46:23.961ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30302024-11-14T04:46:23.961ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 7 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30312024-11-14T04:46:23.965ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30322024-11-14T04:46:23.965ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30332024-11-14T04:46:23.970ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30342024-11-14T04:46:23.970ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30352024-11-14T04:46:23.974ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30362024-11-14T04:46:23.974ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30372024-11-14T04:46:23.993ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30382024-11-14T04:46:23.993ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Array []) node_id = 9 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30392024-11-14T04:46:23.998ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30402024-11-14T04:46:23.998ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30412024-11-14T04:46:24.001ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30422024-11-14T04:46:24.001ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30432024-11-14T04:46:24.004ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30442024-11-14T04:46:24.004ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30452024-11-14T04:46:24.011ZINFOnexus (SagaExecutor): allocated regions for 77777d5b-f8f3-4f3c-8c81-75fcbce5d832 is empty, calling volume_hard_delete file = nexus/src/app/sagas/volume_delete.rs:531 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete
30462024-11-14T04:46:24.014ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30472024-11-14T04:46:24.015ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Bool(true)) node_id = 11 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30482024-11-14T04:46:24.019ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30492024-11-14T04:46:24.020ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30502024-11-14T04:46:24.023ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30512024-11-14T04:46:24.023ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Bool(true)) node_id = 12 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30522024-11-14T04:46:24.028ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30532024-11-14T04:46:24.028ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30542024-11-14T04:46:24.032ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30552024-11-14T04:46:24.032ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30562024-11-14T04:46:24.034ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30572024-11-14T04:46:24.034ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
30582024-11-14T04:46:24.073ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete sec_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
30592024-11-14T04:46:24.074ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5 saga_name = disk-delete saga_result = SagaResult { saga_id: d058b523-9a14-427b-91a9-ebb9d4b546a5, saga_log: SagaLog { saga_id: d058b523-9a14-427b-91a9-ebb9d4b546a5, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N006 started, N006 succeeded, N005 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(Object {"block_size": String("Traditional"), "create_image_id": Null, "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("f4ffd013-77da-4445-90b9-b231fcd65cef"), "name": String("disk-55f94a4ba62"), "time_created": String("2024-11-14T04:46:22.521820Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.521820Z")}, "pantry_address": Null, "project_id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2024-11-14T04:46:23.499072Z")}, "size": Number(1073741824), "slot": Null, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}), 1: Succeeded(Null), 2: Succeeded(Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("992dae43-7740-49b5-814b-c5341916fc63"), "silo_user_id": String("7d466341-8999-4ad7-a241-ecff7d0e71de")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}), 3: Succeeded(Null), 4: Succeeded(Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("5e6ec445-eaf0-41a7-b58c-e17e36cb0697"), String("eb182fec-aaab-45c0-bcc0-f443ebe55c2c"), String("b94f4172-ff19-4b0a-931a-7eab974838b6")]}}), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Null), 8: Succeeded(Null), 9: Succeeded(Array []), 10: Succeeded(Null), 11: Succeeded(Bool(true)), 12: Succeeded(Bool(true)), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Bool(true), node_outputs: {"crucible_resources_to_delete": Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("5e6ec445-eaf0-41a7-b58c-e17e36cb0697"), String("eb182fec-aaab-45c0-bcc0-f443ebe55c2c"), String("b94f4172-ff19-4b0a-931a-7eab974838b6")]}}, "deleted_disk": Object {"block_size": String("Traditional"), "create_image_id": Null, "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("f4ffd013-77da-4445-90b9-b231fcd65cef"), "name": String("disk-55f94a4ba62"), "time_created": String("2024-11-14T04:46:22.521820Z"), "time_deleted": Null, "time_modified": String("2024-11-14T04:46:22.521820Z")}, "pantry_address": Null, "project_id": String("eabc1198-2a2f-4781-897a-051c378afa22"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2024-11-14T04:46:23.499072Z")}, "size": Number(1073741824), "slot": Null, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}, "freed_crucible_regions": Array [], "no_result1": Null, "no_result_1": Null, "no_result_2": Null, "no_result_3": Null, "no_result_4": Null, "no_result_5": Null, "params_for_volume_delete_subsaga": Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("992dae43-7740-49b5-814b-c5341916fc63"), "silo_user_id": String("7d466341-8999-4ad7-a241-ecff7d0e71de")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("77777d5b-f8f3-4f3c-8c81-75fcbce5d832")}, "volume_delete_subsaga_no_result": Bool(true), "volume_hard_deleted": Bool(true)} }) }
30602024-11-14T04:46:24.074ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 523355 local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 27d734f7-cb00-4e80-94d8-b15ee0b8009e response_code = 204 uri = /v1/disks/disk-55f94a4ba62?project=proj-a32f5c8c37
30612024-11-14T04:46:24.075ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30622024-11-14T04:46:24.076ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30632024-11-14T04:46:24.077ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
30642024-11-14T04:46:24.079ZDEBGnexus (ExternalAuthn): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = DeviceAuthRequestList result = Ok(())
30652024-11-14T04:46:24.079ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30662024-11-14T04:46:24.079ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30672024-11-14T04:46:24.092ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 17716 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 2fce5ecd-ab86-4438-b284-7331b95b5b87 response_code = 200 uri = /device/auth
30682024-11-14T04:46:24.094ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30692024-11-14T04:46:24.094ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30702024-11-14T04:46:24.095ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30712024-11-14T04:46:24.096ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30722024-11-14T04:46:24.097ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
30732024-11-14T04:46:24.098ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
30742024-11-14T04:46:24.098ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30752024-11-14T04:46:24.098ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30762024-11-14T04:46:24.100ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30772024-11-14T04:46:24.100ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30782024-11-14T04:46:24.101ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30792024-11-14T04:46:24.101ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30802024-11-14T04:46:24.103ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30812024-11-14T04:46:24.103ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30822024-11-14T04:46:24.104ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
30832024-11-14T04:46:24.106ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
30842024-11-14T04:46:24.106ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30852024-11-14T04:46:24.106ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30862024-11-14T04:46:24.107ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
30872024-11-14T04:46:24.108ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
30882024-11-14T04:46:24.108ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30892024-11-14T04:46:24.108ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30902024-11-14T04:46:24.111ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30912024-11-14T04:46:24.111ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30922024-11-14T04:46:24.113ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30932024-11-14T04:46:24.113ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30942024-11-14T04:46:24.114ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30952024-11-14T04:46:24.114ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30962024-11-14T04:46:24.116ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
30972024-11-14T04:46:24.116ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
30982024-11-14T04:46:24.117ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
30992024-11-14T04:46:24.118ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
31002024-11-14T04:46:24.118ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31012024-11-14T04:46:24.118ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31022024-11-14T04:46:24.119ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31032024-11-14T04:46:24.119ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31042024-11-14T04:46:24.121ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31052024-11-14T04:46:24.121ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31062024-11-14T04:46:24.122ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31072024-11-14T04:46:24.123ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
31082024-11-14T04:46:24.123ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31092024-11-14T04:46:24.124ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31102024-11-14T04:46:24.132ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31112024-11-14T04:46:24.132ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31122024-11-14T04:46:24.134ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31132024-11-14T04:46:24.134ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31142024-11-14T04:46:24.137ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /device/confirm
31152024-11-14T04:46:24.138ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = DeviceAuthRequest { parent: Fleet, key: "HJXN-CCMK", lookup_type: ByCompositeId("user_code = \\"HJXN-CCMK\\"") } result = Ok(()) uri = /device/confirm
31162024-11-14T04:46:24.138ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31172024-11-14T04:46:24.138ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31182024-11-14T04:46:24.139ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31192024-11-14T04:46:24.139ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31202024-11-14T04:46:24.141ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31212024-11-14T04:46:24.141ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31222024-11-14T04:46:24.143ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31232024-11-14T04:46:24.143ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31242024-11-14T04:46:24.145ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31252024-11-14T04:46:24.145ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31262024-11-14T04:46:24.147ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /device/confirm
31272024-11-14T04:46:24.148ZDEBGnexus (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(()) uri = /device/confirm
31282024-11-14T04:46:24.148ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31292024-11-14T04:46:24.148ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31302024-11-14T04:46:24.151ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31312024-11-14T04:46:24.151ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31322024-11-14T04:46:24.153ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /device/confirm
31332024-11-14T04:46:24.154ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = DeviceAuthRequest { parent: Fleet, key: "HJXN-CCMK", lookup_type: ByCompositeId("user_code = \\"HJXN-CCMK\\"") } result = Ok(()) uri = /device/confirm
31342024-11-14T04:46:24.154ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31352024-11-14T04:46:24.154ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31362024-11-14T04:46:24.157ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31372024-11-14T04:46:24.157ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31382024-11-14T04:46:24.158ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31392024-11-14T04:46:24.159ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31402024-11-14T04:46:24.161ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /device/confirm
31412024-11-14T04:46:24.161ZDEBGnexus (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(()) uri = /device/confirm
31422024-11-14T04:46:24.161ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f roles = RoleSet { roles: {} } uri = /device/confirm
31432024-11-14T04:46:24.162ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f resource = Database result = Ok(()) uri = /device/confirm
31442024-11-14T04:46:24.181ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87684 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 68d4e1c9-d811-47bd-a8ee-ef37680e509f response_code = 204 uri = /device/confirm
31452024-11-14T04:46:24.182ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31462024-11-14T04:46:24.183ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31472024-11-14T04:46:24.194ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 12152 local_addr = 172.30.2.6:443 method = POST remote_addr = 10.151.2.100:52982 req_id = 359a6299-c9d8-40ef-a607-ce28e552b6e6 response_code = 200 uri = /device/token
31482024-11-14T04:46:24.196ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31492024-11-14T04:46:24.196ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31502024-11-14T04:46:24.198ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31512024-11-14T04:46:24.198ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31522024-11-14T04:46:24.199ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31532024-11-14T04:46:24.200ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
31542024-11-14T04:46:24.200ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31552024-11-14T04:46:24.200ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31562024-11-14T04:46:24.201ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31572024-11-14T04:46:24.202ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31582024-11-14T04:46:24.203ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31592024-11-14T04:46:24.203ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31602024-11-14T04:46:24.205ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31612024-11-14T04:46:24.205ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31622024-11-14T04:46:24.206ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31632024-11-14T04:46:24.207ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
31642024-11-14T04:46:24.207ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31652024-11-14T04:46:24.207ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31662024-11-14T04:46:24.208ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31672024-11-14T04:46:24.209ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
31682024-11-14T04:46:24.209ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31692024-11-14T04:46:24.210ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31702024-11-14T04:46:24.213ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31712024-11-14T04:46:24.213ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31722024-11-14T04:46:24.214ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31732024-11-14T04:46:24.214ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31742024-11-14T04:46:24.216ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31752024-11-14T04:46:24.216ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31762024-11-14T04:46:24.217ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31772024-11-14T04:46:24.218ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31782024-11-14T04:46:24.219ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31792024-11-14T04:46:24.220ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
31802024-11-14T04:46:24.220ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31812024-11-14T04:46:24.220ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31822024-11-14T04:46:24.222ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31832024-11-14T04:46:24.222ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31842024-11-14T04:46:24.223ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
31852024-11-14T04:46:24.223ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31862024-11-14T04:46:24.225ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31872024-11-14T04:46:24.226ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
31882024-11-14T04:46:24.226ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31892024-11-14T04:46:24.226ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31902024-11-14T04:46:24.227ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31912024-11-14T04:46:24.227ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31922024-11-14T04:46:24.229ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31932024-11-14T04:46:24.229ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31942024-11-14T04:46:24.231ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31952024-11-14T04:46:24.231ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31962024-11-14T04:46:24.233ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31972024-11-14T04:46:24.233ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31982024-11-14T04:46:24.235ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
31992024-11-14T04:46:24.236ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32002024-11-14T04:46:24.238ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32012024-11-14T04:46:24.238ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32022024-11-14T04:46:24.240ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32032024-11-14T04:46:24.240ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32042024-11-14T04:46:24.242ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32052024-11-14T04:46:24.245ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") }, key: 64123868-fdd8-44b4-af58-803d800a258a, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32062024-11-14T04:46:24.245ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32072024-11-14T04:46:24.245ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32082024-11-14T04:46:24.248ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32092024-11-14T04:46:24.248ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32102024-11-14T04:46:24.250ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32112024-11-14T04:46:24.250ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32122024-11-14T04:46:24.253ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32132024-11-14T04:46:24.253ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32142024-11-14T04:46:24.254ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32152024-11-14T04:46:24.258ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") }, key: 64123868-fdd8-44b4-af58-803d800a258a, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32162024-11-14T04:46:24.258ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32172024-11-14T04:46:24.258ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32182024-11-14T04:46:24.260ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32192024-11-14T04:46:24.260ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32202024-11-14T04:46:24.264ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32212024-11-14T04:46:24.264ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32222024-11-14T04:46:24.277ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32232024-11-14T04:46:24.277ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32242024-11-14T04:46:24.281ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85519 local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = c3ca9da0-0742-4f6e-952a-a0194bdab2fa response_code = 204 uri = /v1/vpc-subnets/default?project=proj-a32f5c8c37&vpc=default
32252024-11-14T04:46:24.281ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Signaled
32262024-11-14T04:46:24.282ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
32272024-11-14T04:46:24.282ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32282024-11-14T04:46:24.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32292024-11-14T04:46:24.283ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32302024-11-14T04:46:24.283ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32312024-11-14T04:46:24.283ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32322024-11-14T04:46:24.284ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32332024-11-14T04:46:24.284ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32342024-11-14T04:46:24.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32352024-11-14T04:46:24.284ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32362024-11-14T04:46:24.284ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32372024-11-14T04:46:24.286ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32382024-11-14T04:46:24.286ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32392024-11-14T04:46:24.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32402024-11-14T04:46:24.288ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32412024-11-14T04:46:24.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32422024-11-14T04:46:24.288ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
32432024-11-14T04:46:24.288ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32442024-11-14T04:46:24.288ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32452024-11-14T04:46:24.289ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32462024-11-14T04:46:24.289ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32472024-11-14T04:46:24.291ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32482024-11-14T04:46:24.291ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32492024-11-14T04:46:24.293ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32502024-11-14T04:46:24.293ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32512024-11-14T04:46:24.294ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32522024-11-14T04:46:24.295ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
32532024-11-14T04:46:24.295ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32542024-11-14T04:46:24.295ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32552024-11-14T04:46:24.297ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32562024-11-14T04:46:24.298ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
32572024-11-14T04:46:24.299ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32582024-11-14T04:46:24.299ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32592024-11-14T04:46:24.302ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32602024-11-14T04:46:24.302ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32612024-11-14T04:46:24.303ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32622024-11-14T04:46:24.304ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32632024-11-14T04:46:24.305ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32642024-11-14T04:46:24.305ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32652024-11-14T04:46:24.306ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32662024-11-14T04:46:24.307ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32672024-11-14T04:46:24.308ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32682024-11-14T04:46:24.309ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
32692024-11-14T04:46:24.309ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32702024-11-14T04:46:24.310ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32712024-11-14T04:46:24.311ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32722024-11-14T04:46:24.311ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32732024-11-14T04:46:24.313ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
32742024-11-14T04:46:24.313ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32752024-11-14T04:46:24.314ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32762024-11-14T04:46:24.315ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
32772024-11-14T04:46:24.315ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32782024-11-14T04:46:24.316ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32792024-11-14T04:46:24.317ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32802024-11-14T04:46:24.317ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32812024-11-14T04:46:24.319ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32822024-11-14T04:46:24.319ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32832024-11-14T04:46:24.320ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32842024-11-14T04:46:24.321ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32852024-11-14T04:46:24.323ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32862024-11-14T04:46:24.323ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32872024-11-14T04:46:24.325ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32882024-11-14T04:46:24.326ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32892024-11-14T04:46:24.328ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32902024-11-14T04:46:24.328ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32912024-11-14T04:46:24.331ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32922024-11-14T04:46:24.334ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32932024-11-14T04:46:24.334ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32942024-11-14T04:46:24.334ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32952024-11-14T04:46:24.336ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32962024-11-14T04:46:24.336ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32972024-11-14T04:46:24.339ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
32982024-11-14T04:46:24.339ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
32992024-11-14T04:46:24.341ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33002024-11-14T04:46:24.341ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33012024-11-14T04:46:24.344ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33022024-11-14T04:46:24.346ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33032024-11-14T04:46:24.346ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33042024-11-14T04:46:24.346ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33052024-11-14T04:46:24.348ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33062024-11-14T04:46:24.349ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33072024-11-14T04:46:24.351ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33082024-11-14T04:46:24.352ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33092024-11-14T04:46:24.354ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33102024-11-14T04:46:24.354ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33112024-11-14T04:46:24.356ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33122024-11-14T04:46:24.356ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33132024-11-14T04:46:24.359ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33142024-11-14T04:46:24.359ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33152024-11-14T04:46:24.364ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33162024-11-14T04:46:24.368ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") }, key: fc9cb0b5-8ba0-4acb-8aa7-669d702de768, lookup_type: ById(fc9cb0b5-8ba0-4acb-8aa7-669d702de768) } result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33172024-11-14T04:46:24.368ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33182024-11-14T04:46:24.368ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33192024-11-14T04:46:24.371ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33202024-11-14T04:46:24.371ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33212024-11-14T04:46:24.376ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33222024-11-14T04:46:24.376ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33232024-11-14T04:46:24.378ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33242024-11-14T04:46:24.379ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33252024-11-14T04:46:24.381ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33262024-11-14T04:46:24.381ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33272024-11-14T04:46:24.384ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33282024-11-14T04:46:24.384ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33292024-11-14T04:46:24.388ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33302024-11-14T04:46:24.389ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33312024-11-14T04:46:24.392ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33322024-11-14T04:46:24.394ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") }, key: dea2096d-dbe9-4ffb-9b54-01f8106501ad, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33332024-11-14T04:46:24.394ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-a32f5c8c37
33342024-11-14T04:46:24.394ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-a32f5c8c37
33352024-11-14T04:46:24.414ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131042 local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 2e69a690-0b08-451c-af34-54326a5319d5 response_code = 204 uri = /v1/vpcs/default?project=proj-a32f5c8c37
33362024-11-14T04:46:24.415ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33372024-11-14T04:46:24.415ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33382024-11-14T04:46:24.416ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33392024-11-14T04:46:24.416ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33402024-11-14T04:46:24.423ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
33412024-11-14T04:46:24.423ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33422024-11-14T04:46:24.423ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ed513e3-687f-4aa7-892d-9ff722c6940f", "content-length": "372", "date": "Thu, 14 Nov 2024 04:46:24 GMT"} })
33432024-11-14T04:46:24.423ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33442024-11-14T04:46:24.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33452024-11-14T04:46:24.424ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
33462024-11-14T04:46:24.425ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("token = \\"fdf13c56b306a4b201b051dbdefbea2dd937bf36\\"") } result = Ok(())
33472024-11-14T04:46:24.426ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33482024-11-14T04:46:24.426ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33492024-11-14T04:46:24.427ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33502024-11-14T04:46:24.427ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33512024-11-14T04:46:24.429ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33522024-11-14T04:46:24.429ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33532024-11-14T04:46:24.430ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33542024-11-14T04:46:24.430ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33552024-11-14T04:46:24.432ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
33562024-11-14T04:46:24.433ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
33572024-11-14T04:46:24.433ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33582024-11-14T04:46:24.433ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33592024-11-14T04:46:24.434ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
33602024-11-14T04:46:24.435ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "fdf13c56b306a4b201b051dbdefbea2dd937bf36", lookup_type: ByCompositeId("fdf13c56b306a4b201b051dbdefbea2dd937bf36") } result = Ok(())
33612024-11-14T04:46:24.435ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33622024-11-14T04:46:24.436ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33632024-11-14T04:46:24.439ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33642024-11-14T04:46:24.440ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
33652024-11-14T04:46:24.440ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
33662024-11-14T04:46:24.440ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33672024-11-14T04:46:24.440ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c3723845-40fb-4609-9f9b-3633cc48cbfc", "date": "Thu, 14 Nov 2024 04:46:24 GMT"} })
33682024-11-14T04:46:24.440ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33692024-11-14T04:46:24.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33702024-11-14T04:46:24.441ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33712024-11-14T04:46:24.441ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33722024-11-14T04:46:24.442ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33732024-11-14T04:46:24.442ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33742024-11-14T04:46:24.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33752024-11-14T04:46:24.442ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33762024-11-14T04:46:24.444ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33772024-11-14T04:46:24.444ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33782024-11-14T04:46:24.445ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33792024-11-14T04:46:24.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33802024-11-14T04:46:24.445ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
33812024-11-14T04:46:24.447ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: 7d466341-8999-4ad7-a241-ecff7d0e71de, lookup_type: ById(7d466341-8999-4ad7-a241-ecff7d0e71de) } result = Ok(())
33822024-11-14T04:46:24.447ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33832024-11-14T04:46:24.447ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33842024-11-14T04:46:24.448ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33852024-11-14T04:46:24.448ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
33862024-11-14T04:46:24.448ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33872024-11-14T04:46:24.448ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33882024-11-14T04:46:24.448ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9eab6e86-82ac-49a9-a163-1edca5a97324", "date": "Thu, 14 Nov 2024 04:46:24 GMT"} })
33892024-11-14T04:46:24.449ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
33902024-11-14T04:46:24.449ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33912024-11-14T04:46:24.450ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 168.457098ms iteration = 2 status = {}
33922024-11-14T04:46:24.451ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
33932024-11-14T04:46:24.452ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) } result = Ok(())
33942024-11-14T04:46:24.452ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
33952024-11-14T04:46:24.452ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
33962024-11-14T04:46:24.454ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
33972024-11-14T04:46:24.454ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
33982024-11-14T04:46:24.456ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
33992024-11-14T04:46:24.456ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34002024-11-14T04:46:24.458ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34012024-11-14T04:46:24.459ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34022024-11-14T04:46:24.460ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34032024-11-14T04:46:24.461ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34042024-11-14T04:46:24.463ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34052024-11-14T04:46:24.463ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34062024-11-14T04:46:24.465ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/projects/proj-a32f5c8c37
34072024-11-14T04:46:24.467ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") } result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34082024-11-14T04:46:24.467ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34092024-11-14T04:46:24.468ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34102024-11-14T04:46:24.470ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34112024-11-14T04:46:24.470ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34122024-11-14T04:46:24.473ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34132024-11-14T04:46:24.473ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34142024-11-14T04:46:24.475ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34152024-11-14T04:46:24.475ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34162024-11-14T04:46:24.477ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {(Silo, 992dae43-7740-49b5-814b-c5341916fc63, "admin")} } uri = /v1/projects/proj-a32f5c8c37
34172024-11-14T04:46:24.479ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Project { parent: Silo { parent: Fleet, key: 992dae43-7740-49b5-814b-c5341916fc63, lookup_type: ById(992dae43-7740-49b5-814b-c5341916fc63) }, key: eabc1198-2a2f-4781-897a-051c378afa22, lookup_type: ByName("proj-a32f5c8c37") } result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34182024-11-14T04:46:24.479ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34192024-11-14T04:46:24.479ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34202024-11-14T04:46:24.482ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34212024-11-14T04:46:24.482ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34222024-11-14T04:46:24.484ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34232024-11-14T04:46:24.484ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34242024-11-14T04:46:24.503ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34252024-11-14T04:46:24.503ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34262024-11-14T04:46:24.508ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34272024-11-14T04:46:24.509ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34282024-11-14T04:46:24.517ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34292024-11-14T04:46:24.517ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34302024-11-14T04:46:24.519ZDEBGnexus (dropshot_external): roles actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d roles = RoleSet { roles: {} } uri = /v1/projects/proj-a32f5c8c37
34312024-11-14T04:46:24.519ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 7d466341-8999-4ad7-a241-ecff7d0e71de, silo_id: 992dae43-7740-49b5-814b-c5341916fc63, .. }) actor_id = 7d466341-8999-4ad7-a241-ecff7d0e71de authenticated = true local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d resource = Database result = Ok(()) uri = /v1/projects/proj-a32f5c8c37
34322024-11-14T04:46:24.534ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118985 local_addr = 172.30.2.6:443 method = DELETE remote_addr = 10.151.2.100:52982 req_id = 99c336e7-609a-4ae7-bfce-e3dac62ed28d response_code = 204 uri = /v1/projects/proj-a32f5c8c37
34332024-11-14T04:46:25.254ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:47769
34342024-11-14T04:46:25.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 45678e48-b674-4d4b-8ce2-b55f379d8b00 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
34352024-11-14T04:46:25.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 45678e48-b674-4d4b-8ce2-b55f379d8b00 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
34362024-11-14T04:46:25.264ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9895 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 45678e48-b674-4d4b-8ce2-b55f379d8b00 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
34372024-11-14T04:46:25.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 4990eee9-9356-43ba-914b-bcbd5e1fc02f roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiIxM2RiMjg5ZC1mOWI2LTQxZTktYjNiZC01YWI0ZjMxMDU2OTQifX0%3D
34382024-11-14T04:46:25.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 4990eee9-9356-43ba-914b-bcbd5e1fc02f resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiIxM2RiMjg5ZC1mOWI2LTQxZTktYjNiZC01YWI0ZjMxMDU2OTQifX0%3D
34392024-11-14T04:46:25.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2357 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 4990eee9-9356-43ba-914b-bcbd5e1fc02f response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiIxM2RiMjg5ZC1mOWI2LTQxZTktYjNiZC01YWI0ZjMxMDU2OTQifX0%3D
34402024-11-14T04:46:25.303ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:49209 remote_addr = [fd00:1122:3344:101::d]:55473
34412024-11-14T04:46:25.305ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1870 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 92aabb84-6c78-45c8-93f1-a75c960c6899 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
34422024-11-14T04:46:25.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
34432024-11-14T04:46:25.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34442024-11-14T04:46:25.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
34452024-11-14T04:46:25.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34462024-11-14T04:46:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34472024-11-14T04:46:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34482024-11-14T04:46:25.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34492024-11-14T04:46:25.323ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34502024-11-14T04:46:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34512024-11-14T04:46:25.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34522024-11-14T04:46:25.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34532024-11-14T04:46:25.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34542024-11-14T04:46:25.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34552024-11-14T04:46:25.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34562024-11-14T04:46:25.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34572024-11-14T04:46:25.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
34582024-11-14T04:46:25.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34592024-11-14T04:46:25.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34602024-11-14T04:46:25.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.466023ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34612024-11-14T04:46:25.428ZINFOnexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/tasks/blueprint_load.rs:88 target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
34622024-11-14T04:46:25.428ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 108.296406ms iteration = 2 status = {"enabled":false,"status":"first target blueprint","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z","time_found":"2024-11-14T04:46:25.428502888Z"}
34632024-11-14T04:46:25.428ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
34642024-11-14T04:46:25.428ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
34652024-11-14T04:46:25.428ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
34662024-11-14T04:46:25.428ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34672024-11-14T04:46:25.428ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.38\u{b5}s iteration = 2 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
34682024-11-14T04:46:25.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34692024-11-14T04:46:25.428ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34702024-11-14T04:46:25.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34712024-11-14T04:46:25.429ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34722024-11-14T04:46:25.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34732024-11-14T04:46:25.429ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34742024-11-14T04:46:25.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34752024-11-14T04:46:25.429ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34762024-11-14T04:46:25.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34772024-11-14T04:46:25.431ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34782024-11-14T04:46:25.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34792024-11-14T04:46:25.432ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34802024-11-14T04:46:25.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34812024-11-14T04:46:25.432ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34822024-11-14T04:46:25.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34832024-11-14T04:46:25.434ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34842024-11-14T04:46:25.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34852024-11-14T04:46:25.434ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34862024-11-14T04:46:25.436ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34872024-11-14T04:46:25.436ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34882024-11-14T04:46:25.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34892024-11-14T04:46:25.436ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34902024-11-14T04:46:25.438ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34912024-11-14T04:46:25.438ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34922024-11-14T04:46:25.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34932024-11-14T04:46:25.438ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34942024-11-14T04:46:25.439ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34952024-11-14T04:46:25.439ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34962024-11-14T04:46:25.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34972024-11-14T04:46:25.472ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
34982024-11-14T04:46:25.504ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
34992024-11-14T04:46:25.536ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
35002024-11-14T04:46:25.569ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
35012024-11-14T04:46:25.601ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
35022024-11-14T04:46:26.509ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b0dc572-32bf-40d0-a6af-c49eb3892172", "content-length": "64", "date": "Thu, 14 Nov 2024 04:46:25 GMT"} })
35032024-11-14T04:46:26.509ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35042024-11-14T04:46:26.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35052024-11-14T04:46:26.511ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35062024-11-14T04:46:26.512ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
35072024-11-14T04:46:26.512ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35082024-11-14T04:46:26.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35092024-11-14T04:46:26.532ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca873b37-2699-403e-8506-cfbd81495941", "content-length": "64", "date": "Thu, 14 Nov 2024 04:46:25 GMT"} })
35102024-11-14T04:46:26.532ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35112024-11-14T04:46:26.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35122024-11-14T04:46:26.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35132024-11-14T04:46:26.536ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
35142024-11-14T04:46:26.536ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35152024-11-14T04:46:26.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35162024-11-14T04:46:26.558ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e705973b-4a79-49c1-8ec7-e090cacb912a", "content-length": "64", "date": "Thu, 14 Nov 2024 04:46:25 GMT"} })
35172024-11-14T04:46:26.558ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35182024-11-14T04:46:26.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35192024-11-14T04:46:26.560ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35202024-11-14T04:46:26.561ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
35212024-11-14T04:46:26.561ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35222024-11-14T04:46:26.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35232024-11-14T04:46:26.571ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "acf7bfa5-88e3-4c7b-ada6-513bf84c8fc2", "content-length": "64", "date": "Thu, 14 Nov 2024 04:46:25 GMT"} })
35242024-11-14T04:46:26.571ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35252024-11-14T04:46:26.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35262024-11-14T04:46:26.573ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35272024-11-14T04:46:26.574ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
35282024-11-14T04:46:26.574ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35292024-11-14T04:46:26.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35302024-11-14T04:46:26.586ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee866465-6976-4ed6-b7b3-406e1d4c3a43", "content-length": "64", "date": "Thu, 14 Nov 2024 04:46:26 GMT"} })
35312024-11-14T04:46:26.586ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35322024-11-14T04:46:26.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35332024-11-14T04:46:26.587ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35342024-11-14T04:46:26.588ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
35352024-11-14T04:46:26.588ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35362024-11-14T04:46:26.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35372024-11-14T04:46:26.597ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 1.168504789s iteration = 2 status = {"nsuccess":5}
35382024-11-14T04:46:35.304ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1497 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 337d6673-0e78-4cd3-b7f0-3506c79f05d5 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
35392024-11-14T04:46:35.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
35402024-11-14T04:46:35.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35412024-11-14T04:46:35.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 3 reason = Timeout
35422024-11-14T04:46:35.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35432024-11-14T04:46:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35442024-11-14T04:46:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35452024-11-14T04:46:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35462024-11-14T04:46:35.323ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35472024-11-14T04:46:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35482024-11-14T04:46:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35492024-11-14T04:46:35.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35502024-11-14T04:46:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35512024-11-14T04:46:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35522024-11-14T04:46:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35532024-11-14T04:46:35.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35542024-11-14T04:46:35.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
35552024-11-14T04:46:35.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35562024-11-14T04:46:35.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35572024-11-14T04:46:35.342ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
35582024-11-14T04:46:35.342ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.059869ms iteration = 3 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
35592024-11-14T04:46:35.351ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.952846ms iteration = 3 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35602024-11-14T04:46:45.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
35612024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
35622024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35632024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35642024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
35652024-11-14T04:46:45.320ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
35662024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
35672024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35682024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 4 reason = Timeout
35692024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35702024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35712024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Timeout
35722024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35732024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
35742024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
35752024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
35762024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35772024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
35782024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35792024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
35802024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
35812024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
35822024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
35832024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
35842024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
35852024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35862024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35872024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35882024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
35892024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35902024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35912024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35922024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35932024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35942024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35952024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35962024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35972024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35982024-11-14T04:46:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
35992024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
36002024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
36012024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
36022024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36032024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
36042024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
36052024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
36062024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36072024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36082024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
36092024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
36102024-11-14T04:46:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36112024-11-14T04:46:45.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36122024-11-14T04:46:45.323ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 984, tv_nsec: 95185728 } })
36132024-11-14T04:46:45.323ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
36142024-11-14T04:46:45.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36152024-11-14T04:46:45.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36162024-11-14T04:46:45.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36172024-11-14T04:46:45.326ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36182024-11-14T04:46:45.327ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.050845ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
36192024-11-14T04:46:45.328ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36202024-11-14T04:46:45.328ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36212024-11-14T04:46:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
36222024-11-14T04:46:45.329ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36232024-11-14T04:46:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36242024-11-14T04:46:45.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36252024-11-14T04:46:45.331ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36262024-11-14T04:46:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
36272024-11-14T04:46:45.331ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
36282024-11-14T04:46:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36292024-11-14T04:46:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36302024-11-14T04:46:45.331ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36312024-11-14T04:46:45.331ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36322024-11-14T04:46:45.334ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 984, tv_nsec: 106511606 } })
36332024-11-14T04:46:45.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
36342024-11-14T04:46:45.334ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36352024-11-14T04:46:45.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36362024-11-14T04:46:45.334ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
36372024-11-14T04:46:45.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
36382024-11-14T04:46:45.336ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36392024-11-14T04:46:45.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36402024-11-14T04:46:45.337ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36412024-11-14T04:46:45.337ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
36422024-11-14T04:46:45.337ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36432024-11-14T04:46:45.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36442024-11-14T04:46:45.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36452024-11-14T04:46:45.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36462024-11-14T04:46:45.341ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36472024-11-14T04:46:45.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36482024-11-14T04:46:45.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36492024-11-14T04:46:45.345ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 984, tv_nsec: 117983668 } })
36502024-11-14T04:46:45.346ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
36512024-11-14T04:46:45.347ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36522024-11-14T04:46:45.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36532024-11-14T04:46:45.349ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
36542024-11-14T04:46:45.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36552024-11-14T04:46:45.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36562024-11-14T04:46:45.352ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36572024-11-14T04:46:45.352ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
36582024-11-14T04:46:45.353ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 33.019056ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
36592024-11-14T04:46:45.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
36602024-11-14T04:46:45.355ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36612024-11-14T04:46:45.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36622024-11-14T04:46:45.368ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
36632024-11-14T04:46:45.368ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.596978ms iteration = 4 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
36642024-11-14T04:46:45.382ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36652024-11-14T04:46:45.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36662024-11-14T04:46:45.398ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.886327ms iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
36672024-11-14T04:46:45.399ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36682024-11-14T04:46:45.401ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 80.271264ms iteration = 2 status = {"errors":[],"records_set_to_done":[]}
36692024-11-14T04:46:45.402ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36702024-11-14T04:46:45.402ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36712024-11-14T04:46:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36722024-11-14T04:46:45.408ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36732024-11-14T04:46:45.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36742024-11-14T04:46:45.410ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
36752024-11-14T04:46:45.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36762024-11-14T04:46:45.412ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
36772024-11-14T04:46:45.412ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36782024-11-14T04:46:45.412ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36792024-11-14T04:46:45.412ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
36802024-11-14T04:46:45.412ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36812024-11-14T04:46:45.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36822024-11-14T04:46:45.414ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36832024-11-14T04:46:45.414ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "966f4359-4b17-4fbf-9169-2677959df840", "content-length": "26", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
36842024-11-14T04:46:45.414ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36852024-11-14T04:46:45.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36862024-11-14T04:46:45.415ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36872024-11-14T04:46:45.416ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 94.980921ms iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
36882024-11-14T04:46:45.425ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 105.702761ms iteration = 4 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36892024-11-14T04:46:45.449ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 128.987345ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
36902024-11-14T04:46:45.450ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
36912024-11-14T04:46:45.450ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36922024-11-14T04:46:45.451ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d7f7d3b-29c5-4f3e-9e21-7d7664b2f3c1", "content-length": "26", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
36932024-11-14T04:46:45.451ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36942024-11-14T04:46:45.454ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36952024-11-14T04:46:45.454ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36962024-11-14T04:46:45.454ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
36972024-11-14T04:46:45.454ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 134.087623ms iteration = 2 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
36982024-11-14T04:46:45.500ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36992024-11-14T04:46:45.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37002024-11-14T04:46:45.513ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37012024-11-14T04:46:45.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
37022024-11-14T04:46:45.521ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
37032024-11-14T04:46:45.521ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37042024-11-14T04:46:45.521ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a4ea60d-a6b4-41e4-977f-02df0ee2d03d", "content-length": "26", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37052024-11-14T04:46:45.521ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37062024-11-14T04:46:45.523ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
37072024-11-14T04:46:45.531ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 211.904407ms iteration = 4 status = {"physical_disks_added":0}
37082024-11-14T04:46:45.556ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
37092024-11-14T04:46:45.557ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f35d939-9088-42b6-b2df-d46850c52a1b", "content-length": "1", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37102024-11-14T04:46:45.557ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 237.265531ms iteration = 2 status = {}
37112024-11-14T04:46:45.558ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
37122024-11-14T04:46:45.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
37132024-11-14T04:46:45.561ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37142024-11-14T04:46:45.561ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 241.614917ms iteration = 2 status = null
37152024-11-14T04:46:45.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37162024-11-14T04:46:45.563ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37172024-11-14T04:46:45.565ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37182024-11-14T04:46:45.565ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37192024-11-14T04:46:45.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37202024-11-14T04:46:45.566ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 245.414757ms iteration = 2 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
37212024-11-14T04:46:45.567ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37222024-11-14T04:46:45.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37232024-11-14T04:46:45.570ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37242024-11-14T04:46:45.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37252024-11-14T04:46:45.605ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37262024-11-14T04:46:45.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37272024-11-14T04:46:45.613ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
37282024-11-14T04:46:45.613ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
37292024-11-14T04:46:45.613ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "98cd8fdc-e219-4457-b9ed-1f40f8658458", "content-length": "372", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37302024-11-14T04:46:45.613ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37312024-11-14T04:46:45.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37322024-11-14T04:46:45.615ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37332024-11-14T04:46:45.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37342024-11-14T04:46:45.617ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37352024-11-14T04:46:45.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
37362024-11-14T04:46:45.619ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37372024-11-14T04:46:45.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37382024-11-14T04:46:45.622ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 302.535211ms iteration = 2 status = 0
37392024-11-14T04:46:45.645ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
37402024-11-14T04:46:45.646ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "36dbf99b-da6e-4d08-bab0-b74e14342131", "content-length": "691", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37412024-11-14T04:46:45.646ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
37422024-11-14T04:46:45.646ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
37432024-11-14T04:46:45.646ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
37442024-11-14T04:46:45.647ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 327.501125ms iteration = 2 status = {}
37452024-11-14T04:46:45.656ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
37462024-11-14T04:46:45.656ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
37472024-11-14T04:46:45.656ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "550a49a2-86b6-4dab-92b4-4e9d73f64404", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37482024-11-14T04:46:45.656ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37492024-11-14T04:46:45.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37502024-11-14T04:46:45.658ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37512024-11-14T04:46:45.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37522024-11-14T04:46:45.662ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37532024-11-14T04:46:45.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37542024-11-14T04:46:45.667ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
37552024-11-14T04:46:45.667ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
37562024-11-14T04:46:45.667ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a968a23f-e2fe-4b21-becf-b20b1b1c4a97", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37572024-11-14T04:46:45.669ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 349.038318ms iteration = 3 status = {}
37582024-11-14T04:46:45.692ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37592024-11-14T04:46:45.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37602024-11-14T04:46:45.701ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
37612024-11-14T04:46:45.701ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37622024-11-14T04:46:45.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37632024-11-14T04:46:45.767ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
37642024-11-14T04:46:45.767ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cb29879-b8ad-4a1e-8be6-59cdd7946f0f", "content-length": "134", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37652024-11-14T04:46:45.767ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
37662024-11-14T04:46:45.767ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
37672024-11-14T04:46:45.769ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {}}
37682024-11-14T04:46:45.769ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
37692024-11-14T04:46:45.769ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37702024-11-14T04:46:45.769ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {}
37712024-11-14T04:46:45.769ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37722024-11-14T04:46:45.769ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37732024-11-14T04:46:45.769ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
37742024-11-14T04:46:45.769ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37752024-11-14T04:46:45.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37762024-11-14T04:46:45.773ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
37772024-11-14T04:46:45.774ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa9db070-ef08-45f1-a3ce-3da2b4af4e70", "content-length": "2", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37782024-11-14T04:46:45.774ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
37792024-11-14T04:46:45.775ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f34d1f51-2f9d-4ce3-88f4-6623679063d1", "content-length": "51", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37802024-11-14T04:46:45.777ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
37812024-11-14T04:46:45.777ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
37822024-11-14T04:46:46.121ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b7c50a44-75f9-46e0-98e0-fc0f0acf8833", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37832024-11-14T04:46:46.121ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
37842024-11-14T04:46:46.121ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
37852024-11-14T04:46:46.121ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a1793f3-6a7b-4b36-8484-2e922e2bdc28", "content-length": "682", "date": "Thu, 14 Nov 2024 04:46:45 GMT"} })
37862024-11-14T04:46:46.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37872024-11-14T04:46:46.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37882024-11-14T04:46:46.124ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37892024-11-14T04:46:46.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37902024-11-14T04:46:46.128ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37912024-11-14T04:46:46.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37922024-11-14T04:46:46.130ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37932024-11-14T04:46:46.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37942024-11-14T04:46:46.136ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 816.036041ms iteration = 2 status = {}
37952024-11-14T04:46:49.572ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:39524
37962024-11-14T04:46:49.572ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39524 req_id = 118493d9-78d7-4681-bd0d-8f5680beb319 roles = RoleSet { roles: {} } uri = /metrics/producers
37972024-11-14T04:46:49.572ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39524 req_id = 118493d9-78d7-4681-bd0d-8f5680beb319 resource = Database result = Ok(()) uri = /metrics/producers
37982024-11-14T04:46:49.584ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = e5586e29-e301-4b24-9a2c-39545da75716
37992024-11-14T04:46:49.584ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11688 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39524 req_id = 118493d9-78d7-4681-bd0d-8f5680beb319 response_code = 201 uri = /metrics/producers
38002024-11-14T04:46:50.276ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1288 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 0c4918e6-f601-4cd8-84fe-282ff0689d6b response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
38012024-11-14T04:46:55.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
38022024-11-14T04:46:55.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 5 reason = Timeout
38032024-11-14T04:46:55.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38042024-11-14T04:46:55.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38052024-11-14T04:46:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38062024-11-14T04:46:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38072024-11-14T04:46:55.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38082024-11-14T04:46:55.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38092024-11-14T04:46:55.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38102024-11-14T04:46:55.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.891803ms iteration = 5 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38112024-11-14T04:46:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38122024-11-14T04:46:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38132024-11-14T04:46:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38142024-11-14T04:46:55.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38152024-11-14T04:46:55.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38162024-11-14T04:46:55.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38172024-11-14T04:46:55.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
38182024-11-14T04:46:55.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38192024-11-14T04:46:55.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38202024-11-14T04:46:55.338ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
38212024-11-14T04:46:55.338ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.015002ms iteration = 5 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
38222024-11-14T04:47:00.458ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:35716
38232024-11-14T04:47:00.460ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35716 req_id = 110eb100-9d2b-49ab-b654-a97aad4a1811 roles = RoleSet { roles: {} } uri = /metrics/producers
38242024-11-14T04:47:00.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35716 req_id = 110eb100-9d2b-49ab-b654-a97aad4a1811 resource = Database result = Ok(()) uri = /metrics/producers
38252024-11-14T04:47:00.559ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = f4c8f330-91d6-4976-b637-c72f900d8c87
38262024-11-14T04:47:00.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99207 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35716 req_id = 110eb100-9d2b-49ab-b654-a97aad4a1811 response_code = 201 uri = /metrics/producers
38272024-11-14T04:47:02.378ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:45013
38282024-11-14T04:47:02.378ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45013 req_id = 2fdb924e-d0a3-42be-975e-82fea64dce5a roles = RoleSet { roles: {} } uri = /metrics/producers
38292024-11-14T04:47:02.378ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45013 req_id = 2fdb924e-d0a3-42be-975e-82fea64dce5a resource = Database result = Ok(()) uri = /metrics/producers
38302024-11-14T04:47:02.520ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 18d4ee8d-1d67-44aa-96f9-89dfcadba34d
38312024-11-14T04:47:02.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142181 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45013 req_id = 2fdb924e-d0a3-42be-975e-82fea64dce5a response_code = 201 uri = /metrics/producers
38322024-11-14T04:47:05.282ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1581 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 240237fa-2d9f-409a-9528-b5016d069274 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
38332024-11-14T04:47:05.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
38342024-11-14T04:47:05.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 6 reason = Timeout
38352024-11-14T04:47:05.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38362024-11-14T04:47:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38372024-11-14T04:47:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38382024-11-14T04:47:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38392024-11-14T04:47:05.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38402024-11-14T04:47:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38412024-11-14T04:47:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38422024-11-14T04:47:05.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.861791ms iteration = 6 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38432024-11-14T04:47:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38442024-11-14T04:47:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38452024-11-14T04:47:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38462024-11-14T04:47:05.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38472024-11-14T04:47:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38482024-11-14T04:47:05.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38492024-11-14T04:47:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
38502024-11-14T04:47:05.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38512024-11-14T04:47:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38522024-11-14T04:47:05.343ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
38532024-11-14T04:47:05.343ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.76878ms iteration = 6 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
38542024-11-14T04:47:10.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 9df5215a-3e4a-4477-af8e-7cd65957caf7 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
38552024-11-14T04:47:10.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 9df5215a-3e4a-4477-af8e-7cd65957caf7 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
38562024-11-14T04:47:10.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2687 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 9df5215a-3e4a-4477-af8e-7cd65957caf7 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
38572024-11-14T04:47:10.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 79de151c-2391-4557-8580-6a4e1d06c2c6 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
38582024-11-14T04:47:10.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 79de151c-2391-4557-8580-6a4e1d06c2c6 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
38592024-11-14T04:47:10.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2431 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 79de151c-2391-4557-8580-6a4e1d06c2c6 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
38602024-11-14T04:47:14.315ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:55926
38612024-11-14T04:47:14.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:55926 req_id = 88a1d287-ba28-49f4-b0de-1d38f2a61a7d roles = RoleSet { roles: {} } uri = /metrics/producers
38622024-11-14T04:47:14.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:55926 req_id = 88a1d287-ba28-49f4-b0de-1d38f2a61a7d resource = Database result = Ok(()) uri = /metrics/producers
38632024-11-14T04:47:14.328ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = bc37b1eb-3677-438e-a276-b6d3e6fe3f8e
38642024-11-14T04:47:14.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9952 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:55926 req_id = 88a1d287-ba28-49f4-b0de-1d38f2a61a7d response_code = 201 uri = /metrics/producers
38652024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
38662024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
38672024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Timeout
38682024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38692024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
38702024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38712024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
38722024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
38732024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
38742024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
38752024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38762024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
38772024-11-14T04:47:15.320ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:37:15.320876119 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
38782024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
38792024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
38802024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
38812024-11-14T04:47:15.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
38822024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
38832024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
38842024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
38852024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38862024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 3 reason = Timeout
38872024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
38882024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
38892024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
38902024-11-14T04:47:15.321ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
38912024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Timeout
38922024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
38932024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
38942024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
38952024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
38962024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38972024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
38982024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38992024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
39002024-11-14T04:47:15.321ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
39012024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
39022024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39032024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39042024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
39052024-11-14T04:47:15.321ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
39062024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.21\u{b5}s iteration = 3 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
39072024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
39082024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
39092024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39102024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
39112024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39122024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
39132024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
39142024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39152024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
39162024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
39172024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
39182024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
39192024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
39202024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
39212024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
39222024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
39232024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
39242024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
39252024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39262024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
39272024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
39282024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39292024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
39302024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39312024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
39322024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
39332024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39342024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39352024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39362024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
39372024-11-14T04:47:15.321ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1014, tv_nsec: 93568507 } })
39382024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39392024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39402024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
39412024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39422024-11-14T04:47:15.321ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1014, tv_nsec: 93463004 } })
39432024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
39442024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39452024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
39462024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
39472024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39482024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
39492024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39502024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
39512024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
39522024-11-14T04:47:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39532024-11-14T04:47:15.322ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
39542024-11-14T04:47:15.322ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.440752ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
39552024-11-14T04:47:15.322ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
39562024-11-14T04:47:15.322ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.324589ms iteration = 3 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
39572024-11-14T04:47:15.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39582024-11-14T04:47:15.324ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.625596ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
39592024-11-14T04:47:15.324ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
39602024-11-14T04:47:15.324ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
39612024-11-14T04:47:15.324ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39622024-11-14T04:47:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
39632024-11-14T04:47:15.325ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39642024-11-14T04:47:15.326ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:37:15.320876119 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
39652024-11-14T04:47:15.326ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.19272ms iteration = 2 status = {"expiration":"2024-11-14T04:37:15.320876119Z","pruned":[]}
39662024-11-14T04:47:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39672024-11-14T04:47:15.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39682024-11-14T04:47:15.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
39692024-11-14T04:47:15.328ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39702024-11-14T04:47:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39712024-11-14T04:47:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39722024-11-14T04:47:15.329ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39732024-11-14T04:47:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39742024-11-14T04:47:15.331ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39752024-11-14T04:47:15.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39762024-11-14T04:47:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
39772024-11-14T04:47:15.333ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39782024-11-14T04:47:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39792024-11-14T04:47:15.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39802024-11-14T04:47:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39812024-11-14T04:47:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39822024-11-14T04:47:15.336ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39832024-11-14T04:47:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39842024-11-14T04:47:15.336ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39852024-11-14T04:47:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
39862024-11-14T04:47:15.339ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39872024-11-14T04:47:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39882024-11-14T04:47:15.348ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1014, tv_nsec: 120079355 } })
39892024-11-14T04:47:15.348ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
39902024-11-14T04:47:15.350ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39912024-11-14T04:47:15.350ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39922024-11-14T04:47:15.350ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 29.216106ms iteration = 3 status = {"errors":[],"records_set_to_done":[]}
39932024-11-14T04:47:15.353ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39942024-11-14T04:47:15.353ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39952024-11-14T04:47:15.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
39962024-11-14T04:47:15.353ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
39972024-11-14T04:47:15.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39982024-11-14T04:47:15.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
39992024-11-14T04:47:15.355ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40002024-11-14T04:47:15.356ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1014, tv_nsec: 127963683 } })
40012024-11-14T04:47:15.356ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
40022024-11-14T04:47:15.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
40032024-11-14T04:47:15.359ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40042024-11-14T04:47:15.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40052024-11-14T04:47:15.362ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40062024-11-14T04:47:15.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40072024-11-14T04:47:15.366ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
40082024-11-14T04:47:15.366ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
40092024-11-14T04:47:15.366ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 45.285792ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
40102024-11-14T04:47:15.374ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40112024-11-14T04:47:15.374ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
40122024-11-14T04:47:15.374ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40132024-11-14T04:47:15.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40142024-11-14T04:47:15.376ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40152024-11-14T04:47:15.376ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40162024-11-14T04:47:15.377ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 55.199788ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
40172024-11-14T04:47:15.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40182024-11-14T04:47:15.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40192024-11-14T04:47:15.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40202024-11-14T04:47:15.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40212024-11-14T04:47:15.388ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1014, tv_nsec: 160506996 } })
40222024-11-14T04:47:15.388ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
40232024-11-14T04:47:15.391ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40242024-11-14T04:47:15.391ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40252024-11-14T04:47:15.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40262024-11-14T04:47:15.393ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40272024-11-14T04:47:15.393ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40282024-11-14T04:47:15.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40292024-11-14T04:47:15.396ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40302024-11-14T04:47:15.396ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40312024-11-14T04:47:15.397ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 76.198857ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40322024-11-14T04:47:15.398ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40332024-11-14T04:47:15.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40342024-11-14T04:47:15.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40352024-11-14T04:47:15.399ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40362024-11-14T04:47:15.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40372024-11-14T04:47:15.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
40382024-11-14T04:47:15.400ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40392024-11-14T04:47:15.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40402024-11-14T04:47:15.421ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40412024-11-14T04:47:15.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40422024-11-14T04:47:15.425ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 103.710583ms iteration = 3 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
40432024-11-14T04:47:15.427ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40442024-11-14T04:47:15.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
40452024-11-14T04:47:15.433ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 112.059175ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
40462024-11-14T04:47:15.450ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40472024-11-14T04:47:15.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40482024-11-14T04:47:15.455ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 133.681931ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
40492024-11-14T04:47:15.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40502024-11-14T04:47:15.458ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
40512024-11-14T04:47:15.459ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
40522024-11-14T04:47:15.459ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
40532024-11-14T04:47:15.459ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40542024-11-14T04:47:15.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40552024-11-14T04:47:15.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40562024-11-14T04:47:15.460ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40572024-11-14T04:47:15.461ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40582024-11-14T04:47:15.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40592024-11-14T04:47:15.463ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40602024-11-14T04:47:15.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40612024-11-14T04:47:15.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40622024-11-14T04:47:15.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40632024-11-14T04:47:15.466ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
40642024-11-14T04:47:15.466ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
40652024-11-14T04:47:15.466ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 145.405221ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
40662024-11-14T04:47:15.467ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
40672024-11-14T04:47:15.467ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
40682024-11-14T04:47:15.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40692024-11-14T04:47:15.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
40702024-11-14T04:47:15.467ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40712024-11-14T04:47:15.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40722024-11-14T04:47:15.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40732024-11-14T04:47:15.468ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40742024-11-14T04:47:15.469ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 148.534811ms iteration = 3 status = 0
40752024-11-14T04:47:15.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40762024-11-14T04:47:15.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40772024-11-14T04:47:15.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40782024-11-14T04:47:15.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40792024-11-14T04:47:15.473ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40802024-11-14T04:47:15.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40812024-11-14T04:47:15.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
40822024-11-14T04:47:15.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40832024-11-14T04:47:15.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40842024-11-14T04:47:15.474ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40852024-11-14T04:47:15.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40862024-11-14T04:47:15.477ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
40872024-11-14T04:47:15.483ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
40882024-11-14T04:47:15.483ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 162.975509ms iteration = 7 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
40892024-11-14T04:47:15.492ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
40902024-11-14T04:47:15.492ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 171.582819ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
40912024-11-14T04:47:15.493ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
40922024-11-14T04:47:15.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
40932024-11-14T04:47:15.496ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40942024-11-14T04:47:15.496ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
40952024-11-14T04:47:15.496ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 175.698199ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
40962024-11-14T04:47:15.512ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
40972024-11-14T04:47:15.512ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40982024-11-14T04:47:15.512ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "48f0da15-9f3a-4e47-9f4f-a98924ef7394", "content-length": "26", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
40992024-11-14T04:47:15.512ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41002024-11-14T04:47:15.513ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41012024-11-14T04:47:15.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41022024-11-14T04:47:15.514ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41032024-11-14T04:47:15.517ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41042024-11-14T04:47:15.519ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
41052024-11-14T04:47:15.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41062024-11-14T04:47:15.519ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41072024-11-14T04:47:15.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41082024-11-14T04:47:15.531ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 210.534647ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
41092024-11-14T04:47:15.542ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
41102024-11-14T04:47:15.542ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
41112024-11-14T04:47:15.542ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
41122024-11-14T04:47:15.542ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 221.376321ms iteration = 3 status = {"generation":2}
41132024-11-14T04:47:15.544ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
41142024-11-14T04:47:15.544ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41152024-11-14T04:47:15.545ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f88ded60-e243-46c8-856c-5f67ded509de", "content-length": "26", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
41162024-11-14T04:47:15.545ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41172024-11-14T04:47:15.547ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
41182024-11-14T04:47:15.556ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
41192024-11-14T04:47:15.556ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
41202024-11-14T04:47:15.556ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
41212024-11-14T04:47:15.556ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41222024-11-14T04:47:15.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41232024-11-14T04:47:15.573ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41242024-11-14T04:47:15.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41252024-11-14T04:47:15.578ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
41262024-11-14T04:47:15.578ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41272024-11-14T04:47:15.579ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa82e590-e6e1-4654-8fa5-4f64890bf4c1", "content-length": "26", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
41282024-11-14T04:47:15.579ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41292024-11-14T04:47:15.581ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41302024-11-14T04:47:15.586ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(2)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(2)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
41312024-11-14T04:47:15.586ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 265.879489ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
41322024-11-14T04:47:15.592ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 271.253255ms iteration = 5 status = {"physical_disks_added":0}
41332024-11-14T04:47:15.629ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
41342024-11-14T04:47:15.629ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 308.855625ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
41352024-11-14T04:47:15.684ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
41362024-11-14T04:47:15.687ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
41372024-11-14T04:47:15.688ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "50c22b78-5025-40bb-9362-5726ba8ccc81", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41382024-11-14T04:47:15.688ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41392024-11-14T04:47:15.689ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
41402024-11-14T04:47:15.690ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "197e4eb4-7fca-4011-b318-0b839600f238", "content-length": "1", "date": "Thu, 14 Nov 2024 04:47:14 GMT"} })
41412024-11-14T04:47:15.690ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2e3bc401-e02f-4da4-a798-f98991f46d43", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41422024-11-14T04:47:15.690ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41432024-11-14T04:47:15.695ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
41442024-11-14T04:47:15.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41452024-11-14T04:47:15.696ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41462024-11-14T04:47:15.696ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 375.69579ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
41472024-11-14T04:47:15.696ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
41482024-11-14T04:47:15.696ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41492024-11-14T04:47:15.697ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bcb3f43-e864-4f23-b132-5be3bfe6adc9", "content-length": "372", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
41502024-11-14T04:47:15.697ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41512024-11-14T04:47:15.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41522024-11-14T04:47:15.698ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41532024-11-14T04:47:15.700ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 379.959093ms iteration = 3 status = null
41542024-11-14T04:47:15.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41552024-11-14T04:47:15.701ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41562024-11-14T04:47:15.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41572024-11-14T04:47:15.704ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
41582024-11-14T04:47:15.704ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
41592024-11-14T04:47:15.704ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41602024-11-14T04:47:15.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41612024-11-14T04:47:15.731ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 410.020064ms iteration = 3 status = {}
41622024-11-14T04:47:15.742ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41632024-11-14T04:47:15.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41642024-11-14T04:47:15.745ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41652024-11-14T04:47:15.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41662024-11-14T04:47:15.746ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
41672024-11-14T04:47:15.746ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
41682024-11-14T04:47:15.746ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b37605e9-805b-4aef-a9d4-bb1b0767caa0", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
41692024-11-14T04:47:15.746ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41702024-11-14T04:47:15.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41712024-11-14T04:47:15.748ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
41722024-11-14T04:47:15.748ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
41732024-11-14T04:47:15.748ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
41742024-11-14T04:47:15.748ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 426.954064ms iteration = 3 status = {"generation":1}
41752024-11-14T04:47:15.749ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41762024-11-14T04:47:15.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41772024-11-14T04:47:15.753ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41782024-11-14T04:47:15.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41792024-11-14T04:47:15.756ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
41802024-11-14T04:47:15.756ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41812024-11-14T04:47:15.756ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "93293639-d5ac-4a30-8f0a-dcc8a485654b", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
41822024-11-14T04:47:15.758ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 437.720376ms iteration = 4 status = {}
41832024-11-14T04:47:15.782ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
41842024-11-14T04:47:15.783ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "4f387d7a-62e1-4fe5-be27-ad838623e361", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
41852024-11-14T04:47:15.783ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
41862024-11-14T04:47:15.785ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d6ce98f6-4fc6-473b-bad6-ef03d3366d1a", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
41872024-11-14T04:47:15.785ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
41882024-11-14T04:47:15.786ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d09ed43b-2d29-41eb-ba5d-1f69037aa37e", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
41892024-11-14T04:47:15.786ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
41902024-11-14T04:47:15.788ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
41912024-11-14T04:47:15.788ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 467.424726ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
41922024-11-14T04:47:15.805ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41932024-11-14T04:47:15.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41942024-11-14T04:47:15.807ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41952024-11-14T04:47:15.807ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41962024-11-14T04:47:15.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41972024-11-14T04:47:15.810ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
41982024-11-14T04:47:15.810ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41992024-11-14T04:47:15.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42002024-11-14T04:47:15.811ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42012024-11-14T04:47:15.813ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42022024-11-14T04:47:15.813ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42032024-11-14T04:47:15.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42042024-11-14T04:47:15.908ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
42052024-11-14T04:47:15.908ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42062024-11-14T04:47:15.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42072024-11-14T04:47:15.909ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
42082024-11-14T04:47:15.918ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
42092024-11-14T04:47:15.919ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f09d977-9b56-4a1c-a823-7e94f3c56868", "content-length": "134", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
42102024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
42112024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
42122024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
42132024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
42142024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42152024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42162024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42172024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42182024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42192024-11-14T04:47:15.919ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42202024-11-14T04:47:15.920ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
42212024-11-14T04:47:15.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42222024-11-14T04:47:15.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42232024-11-14T04:47:15.921ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
42242024-11-14T04:47:15.921ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d5c8ff6-3cca-4fa6-891d-cf148fbd191e", "content-length": "2", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
42252024-11-14T04:47:15.921ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
42262024-11-14T04:47:15.921ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "40325eb2-93af-44e0-89d7-35be3b2d1bcc", "content-length": "51", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
42272024-11-14T04:47:15.921ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
42282024-11-14T04:47:15.921ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
42292024-11-14T04:47:15.979ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "35cfc44e-0798-4a3d-8ea2-ef33f4d24f05", "content-length": "691", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
42302024-11-14T04:47:15.979ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
42312024-11-14T04:47:15.979ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
42322024-11-14T04:47:15.979ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
42332024-11-14T04:47:15.980ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 659.306221ms iteration = 3 status = {}
42342024-11-14T04:47:16.073ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7a04f946-0fb2-49da-8bc2-61cbe1d3b4d3", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
42352024-11-14T04:47:16.073ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
42362024-11-14T04:47:16.073ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
42372024-11-14T04:47:16.074ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f10810b6-fc36-441c-8f08-5a8266c5f72f", "content-length": "682", "date": "Thu, 14 Nov 2024 04:47:15 GMT"} })
42382024-11-14T04:47:16.074ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42392024-11-14T04:47:16.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42402024-11-14T04:47:16.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42412024-11-14T04:47:16.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42422024-11-14T04:47:16.278ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42432024-11-14T04:47:16.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42442024-11-14T04:47:16.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42452024-11-14T04:47:16.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42462024-11-14T04:47:16.397ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.076856854s iteration = 3 status = {}
42472024-11-14T04:47:18.950ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:62720
42482024-11-14T04:47:18.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:62720 req_id = b48a0907-df84-4168-85f2-2d7b2879a45a roles = RoleSet { roles: {} } uri = /metrics/producers
42492024-11-14T04:47:18.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:62720 req_id = b48a0907-df84-4168-85f2-2d7b2879a45a resource = Database result = Ok(()) uri = /metrics/producers
42502024-11-14T04:47:18.991ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = c073d51c-abaa-48df-abb6-2f053e9c1bec
42512024-11-14T04:47:18.991ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41244 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:62720 req_id = b48a0907-df84-4168-85f2-2d7b2879a45a response_code = 201 uri = /metrics/producers
42522024-11-14T04:47:20.262ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1093 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = a468df1d-17c6-49cc-930b-d62c1353b2a4 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
42532024-11-14T04:47:25.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 5de94005-1574-42dc-93a4-27ccd26c1624 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
42542024-11-14T04:47:25.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 5de94005-1574-42dc-93a4-27ccd26c1624 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
42552024-11-14T04:47:25.258ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3498 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 5de94005-1574-42dc-93a4-27ccd26c1624 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
42562024-11-14T04:47:25.258ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 3a57f0fd-7e17-492e-8e49-f43812a1bfe8 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
42572024-11-14T04:47:25.258ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 3a57f0fd-7e17-492e-8e49-f43812a1bfe8 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
42582024-11-14T04:47:25.261ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2883 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 3a57f0fd-7e17-492e-8e49-f43812a1bfe8 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
42592024-11-14T04:47:25.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
42602024-11-14T04:47:25.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
42612024-11-14T04:47:25.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42622024-11-14T04:47:25.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42632024-11-14T04:47:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42642024-11-14T04:47:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42652024-11-14T04:47:25.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42662024-11-14T04:47:25.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42672024-11-14T04:47:25.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42682024-11-14T04:47:25.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42692024-11-14T04:47:25.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42702024-11-14T04:47:25.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42712024-11-14T04:47:25.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42722024-11-14T04:47:25.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
42732024-11-14T04:47:25.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42742024-11-14T04:47:25.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42752024-11-14T04:47:25.356ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42762024-11-14T04:47:25.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42772024-11-14T04:47:25.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.742503ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42782024-11-14T04:47:25.445ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
42792024-11-14T04:47:25.445ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 125.408832ms iteration = 8 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
42802024-11-14T04:47:35.263ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1056 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = db73e79a-3a4e-4b0e-8b44-7e0e6b7c7953 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
42812024-11-14T04:47:35.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
42822024-11-14T04:47:35.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42832024-11-14T04:47:35.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
42842024-11-14T04:47:35.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42852024-11-14T04:47:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42862024-11-14T04:47:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42872024-11-14T04:47:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42882024-11-14T04:47:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42892024-11-14T04:47:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42902024-11-14T04:47:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42912024-11-14T04:47:35.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42922024-11-14T04:47:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42932024-11-14T04:47:35.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42942024-11-14T04:47:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
42952024-11-14T04:47:35.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42962024-11-14T04:47:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42972024-11-14T04:47:35.346ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
42982024-11-14T04:47:35.346ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.59453ms iteration = 9 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
42992024-11-14T04:47:35.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43002024-11-14T04:47:35.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43012024-11-14T04:47:35.356ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.017793ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43022024-11-14T04:47:40.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 0d636fef-b041-45c6-ad9e-4e63e1e7fcda roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
43032024-11-14T04:47:40.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 0d636fef-b041-45c6-ad9e-4e63e1e7fcda resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
43042024-11-14T04:47:40.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2977 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 0d636fef-b041-45c6-ad9e-4e63e1e7fcda response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
43052024-11-14T04:47:40.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 8d0f12a6-82ef-4c86-8462-6eeb7a37da6f roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
43062024-11-14T04:47:40.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 8d0f12a6-82ef-4c86-8462-6eeb7a37da6f resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
43072024-11-14T04:47:40.353ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95941 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 8d0f12a6-82ef-4c86-8462-6eeb7a37da6f response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
43082024-11-14T04:47:41.422ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:60026
43092024-11-14T04:47:41.424ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60026 req_id = 43b3a662-6009-4302-89cc-63403afe19ea roles = RoleSet { roles: {} } uri = /metrics/producers
43102024-11-14T04:47:41.424ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60026 req_id = 43b3a662-6009-4302-89cc-63403afe19ea resource = Database result = Ok(()) uri = /metrics/producers
43112024-11-14T04:47:41.561ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = b584ea61-acb3-47a3-a173-0d03f6cae9c9
43122024-11-14T04:47:41.561ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 137424 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60026 req_id = 43b3a662-6009-4302-89cc-63403afe19ea response_code = 201 uri = /metrics/producers
43132024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
43142024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
43152024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43162024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
43172024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
43182024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
43192024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
43202024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43212024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43222024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
43232024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43242024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Timeout
43252024-11-14T04:47:45.320ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
43262024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
43272024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43282024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43292024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 4 reason = Timeout
43302024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
43312024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
43322024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43332024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
43342024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
43352024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
43362024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43372024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
43382024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
43392024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
43402024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43412024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43422024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
43432024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43442024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
43452024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
43462024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43472024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
43482024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43492024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43502024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43512024-11-14T04:47:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
43522024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
43532024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
43542024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
43552024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
43562024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
43572024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
43582024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
43592024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
43602024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
43612024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
43622024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
43632024-11-14T04:47:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
43642024-11-14T04:47:45.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43652024-11-14T04:47:45.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43662024-11-14T04:47:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43672024-11-14T04:47:45.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43682024-11-14T04:47:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43692024-11-14T04:47:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43702024-11-14T04:47:45.331ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43712024-11-14T04:47:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43722024-11-14T04:47:45.332ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43732024-11-14T04:47:45.335ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 15.235621ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
43742024-11-14T04:47:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
43752024-11-14T04:47:45.335ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
43762024-11-14T04:47:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
43772024-11-14T04:47:45.338ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43782024-11-14T04:47:45.338ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1044, tv_nsec: 110792771 } })
43792024-11-14T04:47:45.338ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43802024-11-14T04:47:45.339ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
43812024-11-14T04:47:45.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43822024-11-14T04:47:45.341ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1044, tv_nsec: 113389717 } })
43832024-11-14T04:47:45.341ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
43842024-11-14T04:47:45.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
43852024-11-14T04:47:45.342ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43862024-11-14T04:47:45.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43872024-11-14T04:47:45.344ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43882024-11-14T04:47:45.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43892024-11-14T04:47:45.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43902024-11-14T04:47:45.346ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
43912024-11-14T04:47:45.346ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43922024-11-14T04:47:45.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43932024-11-14T04:47:45.349ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43942024-11-14T04:47:45.349ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.944868ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43952024-11-14T04:47:45.349ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43962024-11-14T04:47:45.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43972024-11-14T04:47:45.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43982024-11-14T04:47:45.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43992024-11-14T04:47:45.351ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1044, tv_nsec: 123717616 } })
44002024-11-14T04:47:45.352ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
44012024-11-14T04:47:45.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
44022024-11-14T04:47:45.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44032024-11-14T04:47:45.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44042024-11-14T04:47:45.406ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
44052024-11-14T04:47:45.407ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 86.814774ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
44062024-11-14T04:47:45.408ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44072024-11-14T04:47:45.408ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 87.494364ms iteration = 4 status = {"errors":[],"records_set_to_done":[]}
44082024-11-14T04:47:45.409ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44092024-11-14T04:47:45.409ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
44102024-11-14T04:47:45.409ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44112024-11-14T04:47:45.409ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 87.896136ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
44122024-11-14T04:47:45.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
44132024-11-14T04:47:45.411ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
44142024-11-14T04:47:45.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44152024-11-14T04:47:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44162024-11-14T04:47:45.412ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44172024-11-14T04:47:45.412ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
44182024-11-14T04:47:45.412ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44192024-11-14T04:47:45.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
44202024-11-14T04:47:45.412ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44212024-11-14T04:47:45.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44222024-11-14T04:47:45.413ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44232024-11-14T04:47:45.414ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44242024-11-14T04:47:45.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44252024-11-14T04:47:45.415ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
44262024-11-14T04:47:45.415ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44272024-11-14T04:47:45.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44282024-11-14T04:47:45.416ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44292024-11-14T04:47:45.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
44302024-11-14T04:47:45.419ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44312024-11-14T04:47:45.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44322024-11-14T04:47:45.438ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
44332024-11-14T04:47:45.438ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44342024-11-14T04:47:45.438ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cbc5eca-f569-46d7-be58-0db69c496fa1", "content-length": "26", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
44352024-11-14T04:47:45.438ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44362024-11-14T04:47:45.440ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44372024-11-14T04:47:45.447ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 126.137033ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
44382024-11-14T04:47:45.453ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44392024-11-14T04:47:45.453ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
44402024-11-14T04:47:45.453ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 133.623419ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
44412024-11-14T04:47:45.457ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
44422024-11-14T04:47:45.457ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44432024-11-14T04:47:45.457ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8708589-969d-40bb-969f-03250a43cd21", "content-length": "26", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
44442024-11-14T04:47:45.457ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44452024-11-14T04:47:45.459ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
44462024-11-14T04:47:45.459ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
44472024-11-14T04:47:45.459ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44482024-11-14T04:47:45.459ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "232b6fd1-d2ef-42f6-a8c7-63685962504d", "content-length": "26", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
44492024-11-14T04:47:45.459ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44502024-11-14T04:47:45.461ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44512024-11-14T04:47:45.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
44522024-11-14T04:47:45.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 147.210903ms iteration = 10 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
44532024-11-14T04:47:45.488ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44542024-11-14T04:47:45.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44552024-11-14T04:47:45.513ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44562024-11-14T04:47:45.513ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
44572024-11-14T04:47:45.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
44582024-11-14T04:47:45.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
44592024-11-14T04:47:45.521ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 200.703443ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
44602024-11-14T04:47:45.538ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
44612024-11-14T04:47:45.538ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "17b6ed2e-08ec-444a-a807-9e7a9f06345a", "content-length": "1", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
44622024-11-14T04:47:45.540ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
44632024-11-14T04:47:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
44642024-11-14T04:47:45.542ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 222.139053ms iteration = 6 status = {"physical_disks_added":0}
44652024-11-14T04:47:45.569ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 248.831926ms iteration = 4 status = {}
44662024-11-14T04:47:45.576ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
44672024-11-14T04:47:45.576ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
44682024-11-14T04:47:45.577ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "16bad91e-f59d-476e-9524-59fe701a0fbd", "content-length": "372", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
44692024-11-14T04:47:45.577ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44702024-11-14T04:47:45.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44712024-11-14T04:47:45.582ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 262.503841ms iteration = 4 status = null
44722024-11-14T04:47:45.596ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44732024-11-14T04:47:45.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44742024-11-14T04:47:45.599ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44752024-11-14T04:47:45.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44762024-11-14T04:47:45.601ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44772024-11-14T04:47:45.603ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44782024-11-14T04:47:45.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
44792024-11-14T04:47:45.603ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44802024-11-14T04:47:45.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44812024-11-14T04:47:45.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44822024-11-14T04:47:45.605ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44832024-11-14T04:47:45.606ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 286.308912ms iteration = 4 status = 0
44842024-11-14T04:47:45.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
44852024-11-14T04:47:45.607ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44862024-11-14T04:47:45.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44872024-11-14T04:47:45.608ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44882024-11-14T04:47:45.610ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
44892024-11-14T04:47:45.610ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44902024-11-14T04:47:45.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44912024-11-14T04:47:45.616ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 294.805478ms iteration = 4 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
44922024-11-14T04:47:45.629ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44932024-11-14T04:47:45.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44942024-11-14T04:47:45.631ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
44952024-11-14T04:47:45.631ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44962024-11-14T04:47:45.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44972024-11-14T04:47:45.644ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
44982024-11-14T04:47:45.644ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2cceb43-2e1e-4597-bb46-f0bf523f8539", "content-length": "134", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
44992024-11-14T04:47:45.644ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
45002024-11-14T04:47:45.644ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
45012024-11-14T04:47:45.644ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
45022024-11-14T04:47:45.644ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
45032024-11-14T04:47:45.644ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45042024-11-14T04:47:45.644ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45052024-11-14T04:47:45.644ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45062024-11-14T04:47:45.644ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
45072024-11-14T04:47:45.645ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45082024-11-14T04:47:45.645ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
45092024-11-14T04:47:45.645ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
45102024-11-14T04:47:45.645ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45112024-11-14T04:47:45.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45122024-11-14T04:47:45.646ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
45132024-11-14T04:47:45.646ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c40d5881-2a5e-4b5e-af5d-9c2c237c88ec", "content-length": "2", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
45142024-11-14T04:47:45.646ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
45152024-11-14T04:47:45.646ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "42ad1061-6ad2-4607-898a-aec2db38b619", "content-length": "51", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
45162024-11-14T04:47:45.646ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
45172024-11-14T04:47:45.646ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
45182024-11-14T04:47:45.704ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
45192024-11-14T04:47:45.736ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
45202024-11-14T04:47:45.736ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
45212024-11-14T04:47:45.757ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "852702d7-1b9c-442c-9d14-1a7cceb8a5f2", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
45222024-11-14T04:47:45.757ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
45232024-11-14T04:47:45.757ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
45242024-11-14T04:47:45.757ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "05105b5f-0f3d-4dfe-a118-1e3e0f9721d2", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
45252024-11-14T04:47:45.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45262024-11-14T04:47:45.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45272024-11-14T04:47:45.758ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "25b72449-b8b5-4da1-86f8-4c114839d8db", "content-length": "691", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
45282024-11-14T04:47:45.758ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
45292024-11-14T04:47:45.758ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
45302024-11-14T04:47:45.758ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
45312024-11-14T04:47:45.759ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 439.017384ms iteration = 4 status = {}
45322024-11-14T04:47:45.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45332024-11-14T04:47:45.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45342024-11-14T04:47:45.763ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4a97431-3daa-4f8d-ad14-71e492dd5607", "content-length": "682", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
45352024-11-14T04:47:45.763ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45362024-11-14T04:47:45.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45372024-11-14T04:47:45.763ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45382024-11-14T04:47:45.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45392024-11-14T04:47:45.769ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
45402024-11-14T04:47:45.769ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
45412024-11-14T04:47:45.769ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3ae760f9-c4c6-4903-90fe-843e117aca70", "date": "Thu, 14 Nov 2024 04:47:45 GMT"} })
45422024-11-14T04:47:45.770ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45432024-11-14T04:47:45.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45442024-11-14T04:47:45.771ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 451.025431ms iteration = 5 status = {}
45452024-11-14T04:47:45.774ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45462024-11-14T04:47:45.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45472024-11-14T04:47:45.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45482024-11-14T04:47:45.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45492024-11-14T04:47:45.880ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 560.095529ms iteration = 4 status = {}
45502024-11-14T04:47:50.355ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1172 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = a7aa5ae6-cf63-4352-9083-b5f5c7e2a696 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
45512024-11-14T04:47:55.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
45522024-11-14T04:47:55.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
45532024-11-14T04:47:55.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45542024-11-14T04:47:55.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45552024-11-14T04:47:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45562024-11-14T04:47:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45572024-11-14T04:47:55.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45582024-11-14T04:47:55.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45592024-11-14T04:47:55.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45602024-11-14T04:47:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45612024-11-14T04:47:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45622024-11-14T04:47:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45632024-11-14T04:47:55.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45642024-11-14T04:47:55.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45652024-11-14T04:47:55.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45662024-11-14T04:47:55.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
45672024-11-14T04:47:55.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45682024-11-14T04:47:55.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45692024-11-14T04:47:55.349ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.368081ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45702024-11-14T04:47:55.448ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
45712024-11-14T04:47:55.448ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 127.988847ms iteration = 11 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
45722024-11-14T04:48:05.261ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1154 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 53b40c62-ea45-4f09-856e-30f17314581e response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
45732024-11-14T04:48:05.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
45742024-11-14T04:48:05.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
45752024-11-14T04:48:05.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45762024-11-14T04:48:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45772024-11-14T04:48:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45782024-11-14T04:48:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45792024-11-14T04:48:05.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45802024-11-14T04:48:05.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45812024-11-14T04:48:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45822024-11-14T04:48:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45832024-11-14T04:48:05.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45842024-11-14T04:48:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45852024-11-14T04:48:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45862024-11-14T04:48:05.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45872024-11-14T04:48:05.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45882024-11-14T04:48:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
45892024-11-14T04:48:05.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45902024-11-14T04:48:05.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45912024-11-14T04:48:05.340ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
45922024-11-14T04:48:05.340ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.583366ms iteration = 12 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
45932024-11-14T04:48:05.352ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.189162ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45942024-11-14T04:48:10.253ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = a1639ee4-29a1-4c7f-aa8c-08a5f3342d46 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
45952024-11-14T04:48:10.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = a1639ee4-29a1-4c7f-aa8c-08a5f3342d46 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
45962024-11-14T04:48:10.256ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2972 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = a1639ee4-29a1-4c7f-aa8c-08a5f3342d46 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
45972024-11-14T04:48:10.256ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = b63de8ce-ef2c-4883-9bc9-4b6a2de2513c roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
45982024-11-14T04:48:10.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = b63de8ce-ef2c-4883-9bc9-4b6a2de2513c resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
45992024-11-14T04:48:10.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2341 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = b63de8ce-ef2c-4883-9bc9-4b6a2de2513c response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
46002024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
46012024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
46022024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
46032024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46042024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
46052024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
46062024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
46072024-11-14T04:48:15.319ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
46082024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
46092024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
46102024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
46112024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
46122024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46132024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46142024-11-14T04:48:15.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
46152024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46162024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
46172024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
46182024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
46192024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46202024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46212024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
46222024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
46232024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
46242024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46252024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46262024-11-14T04:48:15.319ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
46272024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
46282024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.87\u{b5}s iteration = 4 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
46292024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
46302024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
46312024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
46322024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
46332024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46342024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
46352024-11-14T04:48:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
46362024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
46372024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
46382024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
46392024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
46402024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
46412024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
46422024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
46432024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
46442024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
46452024-11-14T04:48:15.320ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:38:15.319944385 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
46462024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
46472024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
46482024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
46492024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
46502024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
46512024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46522024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
46532024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46542024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46552024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46562024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
46572024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46582024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
46592024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46602024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46612024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
46622024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46632024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46642024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46652024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
46662024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
46672024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
46682024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
46692024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
46702024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
46712024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46722024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
46732024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
46742024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
46752024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
46762024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
46772024-11-14T04:48:15.320ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1074, tv_nsec: 92804935 } })
46782024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
46792024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
46802024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
46812024-11-14T04:48:15.320ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1074, tv_nsec: 92815206 } })
46822024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
46832024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
46842024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46852024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
46862024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
46872024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
46882024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
46892024-11-14T04:48:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.102702ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
46902024-11-14T04:48:15.321ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
46912024-11-14T04:48:15.321ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.433932ms iteration = 4 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
46922024-11-14T04:48:15.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46932024-11-14T04:48:15.322ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46942024-11-14T04:48:15.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46952024-11-14T04:48:15.323ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46962024-11-14T04:48:15.323ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.973935ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
46972024-11-14T04:48:15.325ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
46982024-11-14T04:48:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
46992024-11-14T04:48:15.325ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
47002024-11-14T04:48:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
47012024-11-14T04:48:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
47022024-11-14T04:48:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
47032024-11-14T04:48:15.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47042024-11-14T04:48:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47052024-11-14T04:48:15.329ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47062024-11-14T04:48:15.330ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47072024-11-14T04:48:15.331ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1074, tv_nsec: 104062691 } })
47082024-11-14T04:48:15.332ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
47092024-11-14T04:48:15.332ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47102024-11-14T04:48:15.332ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47112024-11-14T04:48:15.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47122024-11-14T04:48:15.333ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47132024-11-14T04:48:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47142024-11-14T04:48:15.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
47152024-11-14T04:48:15.334ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47162024-11-14T04:48:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47172024-11-14T04:48:15.341ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1074, tv_nsec: 114089082 } })
47182024-11-14T04:48:15.342ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
47192024-11-14T04:48:15.345ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:38:15.319944385 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
47202024-11-14T04:48:15.345ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 25.092606ms iteration = 3 status = {"expiration":"2024-11-14T04:38:15.319944385Z","pruned":[]}
47212024-11-14T04:48:15.348ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 28.433124ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
47222024-11-14T04:48:15.353ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47232024-11-14T04:48:15.355ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47242024-11-14T04:48:15.356ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47252024-11-14T04:48:15.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47262024-11-14T04:48:15.364ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47272024-11-14T04:48:15.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
47282024-11-14T04:48:15.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47292024-11-14T04:48:15.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47302024-11-14T04:48:15.373ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47312024-11-14T04:48:15.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47322024-11-14T04:48:15.394ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47332024-11-14T04:48:15.396ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47342024-11-14T04:48:15.398ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
47352024-11-14T04:48:15.398ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47362024-11-14T04:48:15.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47372024-11-14T04:48:15.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
47382024-11-14T04:48:15.399ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47392024-11-14T04:48:15.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47402024-11-14T04:48:15.401ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47412024-11-14T04:48:15.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47422024-11-14T04:48:15.403ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
47432024-11-14T04:48:15.403ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
47442024-11-14T04:48:15.403ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 82.778747ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
47452024-11-14T04:48:15.406ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47462024-11-14T04:48:15.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47472024-11-14T04:48:15.422ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47482024-11-14T04:48:15.427ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
47492024-11-14T04:48:15.428ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47502024-11-14T04:48:15.429ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
47512024-11-14T04:48:15.429ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
47522024-11-14T04:48:15.429ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 109.696027ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
47532024-11-14T04:48:15.430ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 109.904403ms iteration = 5 status = {"errors":[],"records_set_to_done":[]}
47542024-11-14T04:48:15.431ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
47552024-11-14T04:48:15.431ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
47562024-11-14T04:48:15.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
47572024-11-14T04:48:15.433ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 113.118956ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
47582024-11-14T04:48:15.448ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
47592024-11-14T04:48:15.448ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
47602024-11-14T04:48:15.448ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
47612024-11-14T04:48:15.448ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47622024-11-14T04:48:15.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47632024-11-14T04:48:15.448ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 128.935333ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
47642024-11-14T04:48:15.451ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
47652024-11-14T04:48:15.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
47662024-11-14T04:48:15.458ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47672024-11-14T04:48:15.458ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
47682024-11-14T04:48:15.458ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47692024-11-14T04:48:15.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47702024-11-14T04:48:15.466ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
47712024-11-14T04:48:15.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
47722024-11-14T04:48:15.468ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
47732024-11-14T04:48:15.468ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47742024-11-14T04:48:15.470ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d69e432-ba28-4b53-9348-bc52911a32b0", "content-length": "26", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
47752024-11-14T04:48:15.470ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47762024-11-14T04:48:15.471ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47772024-11-14T04:48:15.471ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
47782024-11-14T04:48:15.472ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47792024-11-14T04:48:15.475ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
47802024-11-14T04:48:15.475ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47812024-11-14T04:48:15.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47822024-11-14T04:48:15.489ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
47832024-11-14T04:48:15.498ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
47842024-11-14T04:48:15.498ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
47852024-11-14T04:48:15.498ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
47862024-11-14T04:48:15.498ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47872024-11-14T04:48:15.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47882024-11-14T04:48:15.522ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
47892024-11-14T04:48:15.522ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47902024-11-14T04:48:15.523ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da2d30ca-5adb-4da7-b312-7e628e394151", "content-length": "26", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
47912024-11-14T04:48:15.523ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47922024-11-14T04:48:15.524ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47932024-11-14T04:48:15.536ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47942024-11-14T04:48:15.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47952024-11-14T04:48:15.580ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
47962024-11-14T04:48:15.580ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47972024-11-14T04:48:15.580ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0914ab6b-25e9-4bfc-9e93-0e093a5c9930", "content-length": "372", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
47982024-11-14T04:48:15.580ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47992024-11-14T04:48:15.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48002024-11-14T04:48:15.583ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48012024-11-14T04:48:15.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48022024-11-14T04:48:15.585ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1074, tv_nsec: 357654345 } })
48032024-11-14T04:48:15.585ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
48042024-11-14T04:48:15.587ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 266.639662ms iteration = 5 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
48052024-11-14T04:48:15.589ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48062024-11-14T04:48:15.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48072024-11-14T04:48:15.590ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48082024-11-14T04:48:15.593ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48092024-11-14T04:48:15.593ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48102024-11-14T04:48:15.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48112024-11-14T04:48:15.604ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
48122024-11-14T04:48:15.604ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "6994f843-d761-49da-b9f3-325f699d9167", "content-length": "1", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
48132024-11-14T04:48:15.605ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
48142024-11-14T04:48:15.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
48152024-11-14T04:48:15.613ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 293.448027ms iteration = 5 status = null
48162024-11-14T04:48:15.620ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
48172024-11-14T04:48:15.630ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48182024-11-14T04:48:15.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48192024-11-14T04:48:15.634ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48202024-11-14T04:48:15.634ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48212024-11-14T04:48:15.634ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 314.722984ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
48222024-11-14T04:48:15.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48232024-11-14T04:48:15.634ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48242024-11-14T04:48:15.634ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
48252024-11-14T04:48:15.634ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48262024-11-14T04:48:15.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48272024-11-14T04:48:15.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48282024-11-14T04:48:15.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48292024-11-14T04:48:15.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48302024-11-14T04:48:15.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
48312024-11-14T04:48:15.637ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
48322024-11-14T04:48:15.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
48332024-11-14T04:48:15.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
48342024-11-14T04:48:15.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48352024-11-14T04:48:15.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48362024-11-14T04:48:15.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
48372024-11-14T04:48:15.638ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48382024-11-14T04:48:15.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48392024-11-14T04:48:15.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
48402024-11-14T04:48:15.639ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
48412024-11-14T04:48:15.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
48422024-11-14T04:48:15.640ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48432024-11-14T04:48:15.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48442024-11-14T04:48:15.641ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48452024-11-14T04:48:15.643ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
48462024-11-14T04:48:15.643ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 323.399925ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
48472024-11-14T04:48:15.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
48482024-11-14T04:48:15.645ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48492024-11-14T04:48:15.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48502024-11-14T04:48:15.645ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
48512024-11-14T04:48:15.646ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "cb5573cb-d99a-4519-990f-dc8c32a46bfd", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48522024-11-14T04:48:15.646ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48532024-11-14T04:48:15.647ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "8f33e582-5c58-4b2c-a9d7-7a9b9f08b85a", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48542024-11-14T04:48:15.647ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48552024-11-14T04:48:15.647ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 328.0692ms iteration = 5 status = 0
48562024-11-14T04:48:15.649ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48572024-11-14T04:48:15.649ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 330.097249ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
48582024-11-14T04:48:15.655ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 335.142105ms iteration = 5 status = {}
48592024-11-14T04:48:15.670ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
48602024-11-14T04:48:15.670ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
48612024-11-14T04:48:15.670ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "02971a14-4680-4b00-a587-fa16ea107508", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
48622024-11-14T04:48:15.670ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48632024-11-14T04:48:15.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48642024-11-14T04:48:15.672ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48652024-11-14T04:48:15.672ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
48662024-11-14T04:48:15.672ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
48672024-11-14T04:48:15.672ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
48682024-11-14T04:48:15.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48692024-11-14T04:48:15.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
48702024-11-14T04:48:15.673ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 352.797707ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
48712024-11-14T04:48:15.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48722024-11-14T04:48:15.675ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48732024-11-14T04:48:15.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48742024-11-14T04:48:15.677ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
48752024-11-14T04:48:15.677ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
48762024-11-14T04:48:15.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48772024-11-14T04:48:15.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48782024-11-14T04:48:15.677ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
48792024-11-14T04:48:15.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
48802024-11-14T04:48:15.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
48812024-11-14T04:48:15.679ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48822024-11-14T04:48:15.679ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
48832024-11-14T04:48:15.679ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 359.885362ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
48842024-11-14T04:48:15.679ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
48852024-11-14T04:48:15.680ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
48862024-11-14T04:48:15.680ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
48872024-11-14T04:48:15.680ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 360.040147ms iteration = 4 status = {"generation":2}
48882024-11-14T04:48:15.680ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
48892024-11-14T04:48:15.680ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48902024-11-14T04:48:15.680ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "91240550-d676-46e2-ac82-ba2eff371653", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
48912024-11-14T04:48:15.680ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
48922024-11-14T04:48:15.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
48932024-11-14T04:48:15.682ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 362.549459ms iteration = 6 status = {}
48942024-11-14T04:48:15.704ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
48952024-11-14T04:48:15.704ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
48962024-11-14T04:48:15.704ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48972024-11-14T04:48:15.705ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "32f48cea-de17-48ee-b84c-b34e7e31c8c5", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
48982024-11-14T04:48:15.705ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
48992024-11-14T04:48:15.707ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "1f5c4d6e-b980-4c2d-9c22-2daaaee1a5f8", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
49002024-11-14T04:48:15.707ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
49012024-11-14T04:48:15.707ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c37feba1-9b19-401b-aa28-efc093b3ac96", "content-length": "26", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
49022024-11-14T04:48:15.707ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49032024-11-14T04:48:15.707ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
49042024-11-14T04:48:15.707ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 388.11234ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
49052024-11-14T04:48:15.708ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
49062024-11-14T04:48:15.709ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "8cecd5ff-5017-4070-a028-11ab2c5ff106", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
49072024-11-14T04:48:15.709ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
49082024-11-14T04:48:15.711ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
49092024-11-14T04:48:15.711ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 392.217608ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
49102024-11-14T04:48:15.715ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 395.73016ms iteration = 7 status = {"physical_disks_added":0}
49112024-11-14T04:48:15.720ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49122024-11-14T04:48:15.721ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49132024-11-14T04:48:15.722ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
49142024-11-14T04:48:15.722ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
49152024-11-14T04:48:15.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
49162024-11-14T04:48:15.724ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
49172024-11-14T04:48:15.724ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
49182024-11-14T04:48:15.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
49192024-11-14T04:48:15.726ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
49202024-11-14T04:48:15.726ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
49212024-11-14T04:48:15.726ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
49222024-11-14T04:48:15.726ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 406.857132ms iteration = 4 status = {"generation":1}
49232024-11-14T04:48:15.727ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49242024-11-14T04:48:15.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49252024-11-14T04:48:15.728ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49262024-11-14T04:48:15.730ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49272024-11-14T04:48:15.730ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49282024-11-14T04:48:15.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49292024-11-14T04:48:15.735ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49302024-11-14T04:48:15.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49312024-11-14T04:48:15.736ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49322024-11-14T04:48:15.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49332024-11-14T04:48:15.758ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
49342024-11-14T04:48:15.758ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 438.618222ms iteration = 13 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
49352024-11-14T04:48:15.766ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
49362024-11-14T04:48:15.766ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 446.913712ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
49372024-11-14T04:48:15.821ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49382024-11-14T04:48:15.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49392024-11-14T04:48:15.823ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
49402024-11-14T04:48:15.823ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49412024-11-14T04:48:15.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49422024-11-14T04:48:15.832ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
49432024-11-14T04:48:15.832ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8a3c7a4-cc10-46a6-88fd-6124b6c4508e", "content-length": "134", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
49442024-11-14T04:48:15.832ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
49452024-11-14T04:48:15.832ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
49462024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
49472024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
49482024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49492024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49502024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49512024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
49522024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49532024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
49542024-11-14T04:48:15.833ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
49552024-11-14T04:48:15.833ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49562024-11-14T04:48:15.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49572024-11-14T04:48:15.836ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
49582024-11-14T04:48:15.836ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2efe2cd-ac54-48b9-838d-37e494a7ecd2", "content-length": "2", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
49592024-11-14T04:48:15.836ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
49602024-11-14T04:48:15.836ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "befdad04-b2a9-4cbd-8ed3-a003a25b7853", "content-length": "51", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
49612024-11-14T04:48:15.836ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
49622024-11-14T04:48:15.836ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
49632024-11-14T04:48:15.839ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
49642024-11-14T04:48:15.942ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b423a7ce-3db0-49ad-bc11-bd2d5b3a23be", "content-length": "691", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
49652024-11-14T04:48:15.943ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
49662024-11-14T04:48:15.943ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
49672024-11-14T04:48:15.943ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
49682024-11-14T04:48:15.945ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 625.478262ms iteration = 5 status = {}
49692024-11-14T04:48:16.032ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9dc238bd-5e11-4e75-933f-e9dd6eac7316", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
49702024-11-14T04:48:16.032ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
49712024-11-14T04:48:16.032ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
49722024-11-14T04:48:16.032ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9acf4bdd-93e8-4dfd-bb0e-f5d046c71970", "content-length": "682", "date": "Thu, 14 Nov 2024 04:48:15 GMT"} })
49732024-11-14T04:48:16.032ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49742024-11-14T04:48:16.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49752024-11-14T04:48:16.052ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49762024-11-14T04:48:16.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49772024-11-14T04:48:16.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49782024-11-14T04:48:16.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49792024-11-14T04:48:16.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49802024-11-14T04:48:16.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49812024-11-14T04:48:16.079ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 759.763922ms iteration = 5 status = {}
49822024-11-14T04:48:20.261ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1083 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = ab125d83-23b8-4101-8f7d-be4ac90a6b63 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
49832024-11-14T04:48:25.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
49842024-11-14T04:48:25.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49852024-11-14T04:48:25.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
49862024-11-14T04:48:25.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49872024-11-14T04:48:25.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49882024-11-14T04:48:25.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49892024-11-14T04:48:25.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49902024-11-14T04:48:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49912024-11-14T04:48:25.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49922024-11-14T04:48:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49932024-11-14T04:48:25.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49942024-11-14T04:48:25.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49952024-11-14T04:48:25.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49962024-11-14T04:48:25.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49972024-11-14T04:48:25.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49982024-11-14T04:48:25.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
49992024-11-14T04:48:25.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50002024-11-14T04:48:25.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50012024-11-14T04:48:25.381ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.388226ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50022024-11-14T04:48:25.472ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
50032024-11-14T04:48:25.472ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 153.245168ms iteration = 14 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
50042024-11-14T04:48:35.261ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1082 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = a88b9e58-35b7-4a58-967f-ba59a2495cf4 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
50052024-11-14T04:48:35.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
50062024-11-14T04:48:35.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50072024-11-14T04:48:35.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
50082024-11-14T04:48:35.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50092024-11-14T04:48:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50102024-11-14T04:48:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50112024-11-14T04:48:35.348ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50122024-11-14T04:48:35.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50132024-11-14T04:48:35.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50142024-11-14T04:48:35.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50152024-11-14T04:48:35.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50162024-11-14T04:48:35.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50172024-11-14T04:48:35.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50182024-11-14T04:48:35.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50192024-11-14T04:48:35.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50202024-11-14T04:48:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
50212024-11-14T04:48:35.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50222024-11-14T04:48:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50232024-11-14T04:48:35.379ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.41419ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50242024-11-14T04:48:35.473ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
50252024-11-14T04:48:35.473ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 153.239018ms iteration = 15 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
50262024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
50272024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
50282024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
50292024-11-14T04:48:45.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
50302024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50312024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
50322024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
50332024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50342024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
50352024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50362024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50372024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
50382024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50392024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
50402024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
50412024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
50422024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
50432024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
50442024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
50452024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
50462024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50472024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50482024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
50492024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50502024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
50512024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
50522024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
50532024-11-14T04:48:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
50542024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50552024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50562024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
50572024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50582024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
50592024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50602024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50612024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
50622024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
50632024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50642024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
50652024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
50662024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
50672024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
50682024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
50692024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
50702024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
50712024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
50722024-11-14T04:48:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
50732024-11-14T04:48:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
50742024-11-14T04:48:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
50752024-11-14T04:48:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
50762024-11-14T04:48:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
50772024-11-14T04:48:45.322ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1104, tv_nsec: 94995446 } })
50782024-11-14T04:48:45.322ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
50792024-11-14T04:48:45.322ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50802024-11-14T04:48:45.324ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50812024-11-14T04:48:45.324ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50822024-11-14T04:48:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
50832024-11-14T04:48:45.325ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50842024-11-14T04:48:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50852024-11-14T04:48:45.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
50862024-11-14T04:48:45.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50872024-11-14T04:48:45.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
50882024-11-14T04:48:45.326ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50892024-11-14T04:48:45.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
50902024-11-14T04:48:45.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
50912024-11-14T04:48:45.339ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1104, tv_nsec: 112554284 } })
50922024-11-14T04:48:45.340ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
50932024-11-14T04:48:45.343ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50942024-11-14T04:48:45.343ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1104, tv_nsec: 116077676 } })
50952024-11-14T04:48:45.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
50962024-11-14T04:48:45.343ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
50972024-11-14T04:48:45.385ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50982024-11-14T04:48:45.388ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
50992024-11-14T04:48:45.388ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51002024-11-14T04:48:45.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51012024-11-14T04:48:45.389ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51022024-11-14T04:48:45.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
51032024-11-14T04:48:45.393ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51042024-11-14T04:48:45.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51052024-11-14T04:48:45.397ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51062024-11-14T04:48:45.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51072024-11-14T04:48:45.398ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 77.866495ms iteration = 6 status = {"errors":[],"records_set_to_done":[]}
51082024-11-14T04:48:45.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51092024-11-14T04:48:45.401ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
51102024-11-14T04:48:45.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
51112024-11-14T04:48:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51122024-11-14T04:48:45.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51132024-11-14T04:48:45.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51142024-11-14T04:48:45.404ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 84.150297ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
51152024-11-14T04:48:45.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51162024-11-14T04:48:45.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51172024-11-14T04:48:45.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51182024-11-14T04:48:45.408ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
51192024-11-14T04:48:45.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
51202024-11-14T04:48:45.409ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51212024-11-14T04:48:45.411ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
51222024-11-14T04:48:45.411ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 91.730016ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
51232024-11-14T04:48:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
51242024-11-14T04:48:45.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51252024-11-14T04:48:45.411ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51262024-11-14T04:48:45.411ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
51272024-11-14T04:48:45.411ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51282024-11-14T04:48:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
51292024-11-14T04:48:45.411ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51302024-11-14T04:48:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51312024-11-14T04:48:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51322024-11-14T04:48:45.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51332024-11-14T04:48:45.412ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8103ce16-9ed4-477f-89cc-f8e07a4d7e97", "content-length": "26", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
51342024-11-14T04:48:45.412ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51352024-11-14T04:48:45.413ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51362024-11-14T04:48:45.413ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
51372024-11-14T04:48:45.413ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51382024-11-14T04:48:45.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51392024-11-14T04:48:45.414ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
51402024-11-14T04:48:45.422ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 102.766046ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
51412024-11-14T04:48:45.436ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
51422024-11-14T04:48:45.436ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51432024-11-14T04:48:45.437ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "67b63291-72f9-46b4-a25b-362dcab58fb7", "content-length": "26", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
51442024-11-14T04:48:45.437ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51452024-11-14T04:48:45.440ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
51462024-11-14T04:48:45.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51472024-11-14T04:48:45.450ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
51482024-11-14T04:48:45.450ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51492024-11-14T04:48:45.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51502024-11-14T04:48:45.452ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
51512024-11-14T04:48:45.452ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 132.860967ms iteration = 16 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
51522024-11-14T04:48:45.458ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
51532024-11-14T04:48:45.458ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51542024-11-14T04:48:45.458ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc51d2c2-dc94-4d6d-afb3-80307a3355e7", "content-length": "26", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
51552024-11-14T04:48:45.458ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51562024-11-14T04:48:45.460ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
51572024-11-14T04:48:45.461ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 142.242039ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
51582024-11-14T04:48:45.462ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 141.313202ms iteration = 6 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
51592024-11-14T04:48:45.470ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51602024-11-14T04:48:45.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51612024-11-14T04:48:45.471ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51622024-11-14T04:48:45.471ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
51632024-11-14T04:48:45.471ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51642024-11-14T04:48:45.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51652024-11-14T04:48:45.516ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51662024-11-14T04:48:45.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51672024-11-14T04:48:45.518ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 197.568212ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
51682024-11-14T04:48:45.521ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51692024-11-14T04:48:45.521ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
51702024-11-14T04:48:45.521ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 201.859376ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
51712024-11-14T04:48:45.525ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
51722024-11-14T04:48:45.525ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51732024-11-14T04:48:45.526ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "defdabe8-4691-47cc-9308-10c725463c56", "content-length": "1", "date": "Thu, 14 Nov 2024 04:48:44 GMT"} })
51742024-11-14T04:48:45.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51752024-11-14T04:48:45.527ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
51762024-11-14T04:48:45.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
51772024-11-14T04:48:45.529ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 210.279119ms iteration = 6 status = null
51782024-11-14T04:48:45.547ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 227.379025ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51792024-11-14T04:48:45.557ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 237.504238ms iteration = 8 status = {"physical_disks_added":0}
51802024-11-14T04:48:45.558ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 238.995791ms iteration = 6 status = {}
51812024-11-14T04:48:45.609ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
51822024-11-14T04:48:45.609ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
51832024-11-14T04:48:45.609ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a3f5b8-96c2-469c-800c-f8f79d5ea91f", "content-length": "372", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
51842024-11-14T04:48:45.609ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51852024-11-14T04:48:45.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51862024-11-14T04:48:45.610ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51872024-11-14T04:48:45.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51882024-11-14T04:48:45.612ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51892024-11-14T04:48:45.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
51902024-11-14T04:48:45.614ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51912024-11-14T04:48:45.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51922024-11-14T04:48:45.616ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51932024-11-14T04:48:45.618ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
51942024-11-14T04:48:45.618ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51952024-11-14T04:48:45.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51962024-11-14T04:48:45.650ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
51972024-11-14T04:48:45.650ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
51982024-11-14T04:48:45.741ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "97d03ea5-9b54-498b-83df-97d2305cead9", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
51992024-11-14T04:48:45.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52002024-11-14T04:48:45.741ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52012024-11-14T04:48:45.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52022024-11-14T04:48:45.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52032024-11-14T04:48:45.742ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
52042024-11-14T04:48:45.743ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7024304-105d-4fc7-bf28-a48892659f21", "content-length": "691", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
52052024-11-14T04:48:45.743ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
52062024-11-14T04:48:45.743ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
52072024-11-14T04:48:45.743ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
52082024-11-14T04:48:45.744ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52092024-11-14T04:48:45.744ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52102024-11-14T04:48:45.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52112024-11-14T04:48:45.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52122024-11-14T04:48:45.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52132024-11-14T04:48:45.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52142024-11-14T04:48:45.744ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 425.154252ms iteration = 6 status = {}
52152024-11-14T04:48:45.746ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52162024-11-14T04:48:45.747ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
52172024-11-14T04:48:45.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52182024-11-14T04:48:45.747ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52192024-11-14T04:48:45.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52202024-11-14T04:48:45.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52212024-11-14T04:48:45.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
52222024-11-14T04:48:45.749ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52232024-11-14T04:48:45.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52242024-11-14T04:48:45.751ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 431.226928ms iteration = 6 status = 0
52252024-11-14T04:48:45.754ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
52262024-11-14T04:48:45.754ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
52272024-11-14T04:48:45.754ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "cd31e124-5270-4612-afc2-ac8b8910cfe1", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
52282024-11-14T04:48:45.756ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 436.436309ms iteration = 7 status = {}
52292024-11-14T04:48:45.863ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
52302024-11-14T04:48:45.864ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c9d27ee-03db-4fd1-a524-7b3694fdae1e", "content-length": "134", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
52312024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
52322024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
52332024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
52342024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
52352024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52362024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52372024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52382024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
52392024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52402024-11-14T04:48:45.864ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
52412024-11-14T04:48:45.865ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
52422024-11-14T04:48:45.865ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52432024-11-14T04:48:45.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52442024-11-14T04:48:45.867ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
52452024-11-14T04:48:45.867ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4acd093-87e6-429a-a852-d07e5d2735dd", "content-length": "2", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
52462024-11-14T04:48:45.867ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
52472024-11-14T04:48:45.867ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ba68c7b-b3ab-4b9e-97f7-1e7e58990eca", "content-length": "51", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
52482024-11-14T04:48:45.867ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
52492024-11-14T04:48:45.867ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
52502024-11-14T04:48:45.969ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9d5d7eb8-5673-4b30-b26c-568f5c6d4186", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
52512024-11-14T04:48:45.970ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
52522024-11-14T04:48:45.970ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
52532024-11-14T04:48:45.975ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8eab82c7-9f87-461a-a6d8-ad37ece84cbb", "content-length": "682", "date": "Thu, 14 Nov 2024 04:48:45 GMT"} })
52542024-11-14T04:48:45.975ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52552024-11-14T04:48:45.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52562024-11-14T04:48:45.978ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52572024-11-14T04:48:45.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52582024-11-14T04:48:45.981ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52592024-11-14T04:48:45.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52602024-11-14T04:48:45.983ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52612024-11-14T04:48:45.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52622024-11-14T04:48:45.990ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 671.005891ms iteration = 6 status = {}
52632024-11-14T04:48:49.955ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
52642024-11-14T04:48:49.955ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
52652024-11-14T04:48:49.985ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
52662024-11-14T04:48:49.985ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:62146
52672024-11-14T04:48:49.985ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:62146 req_id = ca2f013b-4001-4762-ba8e-31b8eed2ce19 roles = RoleSet { roles: {} } uri = /metrics/producers
52682024-11-14T04:48:49.986ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:62146 req_id = ca2f013b-4001-4762-ba8e-31b8eed2ce19 resource = Database result = Ok(()) uri = /metrics/producers
52692024-11-14T04:48:50.079ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
52702024-11-14T04:48:50.079ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94055 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:62146 req_id = ca2f013b-4001-4762-ba8e-31b8eed2ce19 response_code = 201 uri = /metrics/producers
52712024-11-14T04:48:50.079ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "ca2f013b-4001-4762-ba8e-31b8eed2ce19", "content-length": "41", "date": "Thu, 14 Nov 2024 04:48:49 GMT"} })
52722024-11-14T04:48:50.081ZDEBGnexus (producer-registration-task): registered with nexus successfully
52732024-11-14T04:48:50.081ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
52742024-11-14T04:48:50.262ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1135 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = e553c1bd-ed95-43c0-b0b4-fc734609ae25 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
52752024-11-14T04:48:55.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 2dd7bad9-7db4-4330-a05a-327c12f84ecd roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
52762024-11-14T04:48:55.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 2dd7bad9-7db4-4330-a05a-327c12f84ecd resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
52772024-11-14T04:48:55.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2865 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 2dd7bad9-7db4-4330-a05a-327c12f84ecd response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
52782024-11-14T04:48:55.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 7a809586-21cd-4338-9e99-566d2f162e7b roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
52792024-11-14T04:48:55.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 7a809586-21cd-4338-9e99-566d2f162e7b resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
52802024-11-14T04:48:55.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2173 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = 7a809586-21cd-4338-9e99-566d2f162e7b response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
52812024-11-14T04:48:55.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
52822024-11-14T04:48:55.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
52832024-11-14T04:48:55.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52842024-11-14T04:48:55.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52852024-11-14T04:48:55.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52862024-11-14T04:48:55.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52872024-11-14T04:48:55.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52882024-11-14T04:48:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52892024-11-14T04:48:55.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52902024-11-14T04:48:55.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.027537ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
52912024-11-14T04:48:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
52922024-11-14T04:48:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52932024-11-14T04:48:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52942024-11-14T04:48:55.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52952024-11-14T04:48:55.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52962024-11-14T04:48:55.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52972024-11-14T04:48:55.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
52982024-11-14T04:48:55.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52992024-11-14T04:48:55.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53002024-11-14T04:48:55.444ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
53012024-11-14T04:48:55.444ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 124.901197ms iteration = 17 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
53022024-11-14T04:49:05.261ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1039 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 52e6136a-4d8d-402a-a354-78d556785eb6 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
53032024-11-14T04:49:05.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
53042024-11-14T04:49:05.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53052024-11-14T04:49:05.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
53062024-11-14T04:49:05.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53072024-11-14T04:49:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53082024-11-14T04:49:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53092024-11-14T04:49:05.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53102024-11-14T04:49:05.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53112024-11-14T04:49:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53122024-11-14T04:49:05.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.49635ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53132024-11-14T04:49:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53142024-11-14T04:49:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53152024-11-14T04:49:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53162024-11-14T04:49:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53172024-11-14T04:49:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53182024-11-14T04:49:05.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53192024-11-14T04:49:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
53202024-11-14T04:49:05.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53212024-11-14T04:49:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53222024-11-14T04:49:05.447ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
53232024-11-14T04:49:05.447ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 128.666276ms iteration = 18 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
53242024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
53252024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
53262024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
53272024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53282024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
53292024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53302024-11-14T04:49:15.319ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
53312024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.25\u{b5}s iteration = 5 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
53322024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
53332024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
53342024-11-14T04:49:15.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
53352024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53362024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
53372024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53382024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
53392024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
53402024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53412024-11-14T04:49:15.319ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:39:15.319085170 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
53422024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
53432024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
53442024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
53452024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53462024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
53472024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
53482024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
53492024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
53502024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
53512024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
53522024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
53532024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53542024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
53552024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53562024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
53572024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
53582024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53592024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
53602024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53612024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
53622024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
53632024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53642024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
53652024-11-14T04:49:15.319ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
53662024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
53672024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
53682024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
53692024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53702024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
53712024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
53722024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
53732024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
53742024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53752024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53762024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53772024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53782024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53792024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53802024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53812024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53822024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53832024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
53842024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
53852024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53862024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
53872024-11-14T04:49:15.319ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1134, tv_nsec: 92616464 } })
53882024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
53892024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53902024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53912024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53922024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53932024-11-14T04:49:15.319ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1134, tv_nsec: 92671036 } })
53942024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53952024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
53962024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
53972024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
53982024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
53992024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
54002024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
54012024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
54022024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
54032024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
54042024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
54052024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
54062024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
54072024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54082024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54092024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
54102024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54112024-11-14T04:49:15.319ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
54122024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
54132024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
54142024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.124583ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
54152024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
54162024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54172024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
54182024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54192024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.079501ms iteration = 5 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
54202024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
54212024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54222024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
54232024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
54242024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54252024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54262024-11-14T04:49:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54272024-11-14T04:49:15.322ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
54282024-11-14T04:49:15.322ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
54292024-11-14T04:49:15.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
54302024-11-14T04:49:15.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
54312024-11-14T04:49:15.322ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54322024-11-14T04:49:15.323ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.870992ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
54332024-11-14T04:49:15.323ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54342024-11-14T04:49:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54352024-11-14T04:49:15.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54362024-11-14T04:49:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
54372024-11-14T04:49:15.325ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54382024-11-14T04:49:15.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54392024-11-14T04:49:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54402024-11-14T04:49:15.326ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54412024-11-14T04:49:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54422024-11-14T04:49:15.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54432024-11-14T04:49:15.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54442024-11-14T04:49:15.327ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:39:15.319085170 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
54452024-11-14T04:49:15.327ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.519196ms iteration = 4 status = {"expiration":"2024-11-14T04:39:15.319085170Z","pruned":[]}
54462024-11-14T04:49:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54472024-11-14T04:49:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54482024-11-14T04:49:15.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54492024-11-14T04:49:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54502024-11-14T04:49:15.330ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1134, tv_nsec: 103291033 } })
54512024-11-14T04:49:15.330ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
54522024-11-14T04:49:15.331ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54532024-11-14T04:49:15.331ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1134, tv_nsec: 104686944 } })
54542024-11-14T04:49:15.332ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
54552024-11-14T04:49:15.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54562024-11-14T04:49:15.332ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54572024-11-14T04:49:15.332ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54582024-11-14T04:49:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
54592024-11-14T04:49:15.335ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
54602024-11-14T04:49:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
54612024-11-14T04:49:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
54622024-11-14T04:49:15.335ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54632024-11-14T04:49:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54642024-11-14T04:49:15.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54652024-11-14T04:49:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54662024-11-14T04:49:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54672024-11-14T04:49:15.336ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54682024-11-14T04:49:15.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
54692024-11-14T04:49:15.336ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54702024-11-14T04:49:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54712024-11-14T04:49:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54722024-11-14T04:49:15.344ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 24.438248ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
54732024-11-14T04:49:15.344ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54742024-11-14T04:49:15.345ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54752024-11-14T04:49:15.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54762024-11-14T04:49:15.347ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
54772024-11-14T04:49:15.347ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54782024-11-14T04:49:15.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54792024-11-14T04:49:15.347ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54802024-11-14T04:49:15.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
54812024-11-14T04:49:15.350ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54822024-11-14T04:49:15.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54832024-11-14T04:49:15.351ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 31.310907ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
54842024-11-14T04:49:15.367ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 48.022561ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54852024-11-14T04:49:15.379ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54862024-11-14T04:49:15.383ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54872024-11-14T04:49:15.383ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54882024-11-14T04:49:15.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54892024-11-14T04:49:15.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54902024-11-14T04:49:15.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54912024-11-14T04:49:15.386ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54922024-11-14T04:49:15.389ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54932024-11-14T04:49:15.390ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54942024-11-14T04:49:15.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54952024-11-14T04:49:15.391ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1134, tv_nsec: 164014891 } })
54962024-11-14T04:49:15.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
54972024-11-14T04:49:15.391ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
54982024-11-14T04:49:15.394ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
54992024-11-14T04:49:15.394ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55002024-11-14T04:49:15.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55012024-11-14T04:49:15.417ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55022024-11-14T04:49:15.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55032024-11-14T04:49:15.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
55042024-11-14T04:49:15.420ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
55052024-11-14T04:49:15.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
55062024-11-14T04:49:15.424ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55072024-11-14T04:49:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
55082024-11-14T04:49:15.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55092024-11-14T04:49:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55102024-11-14T04:49:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55112024-11-14T04:49:15.428ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
55122024-11-14T04:49:15.428ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
55132024-11-14T04:49:15.428ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 108.925374ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
55142024-11-14T04:49:15.431ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 111.726266ms iteration = 7 status = {"errors":[],"records_set_to_done":[]}
55152024-11-14T04:49:15.432ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55162024-11-14T04:49:15.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55172024-11-14T04:49:15.434ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55182024-11-14T04:49:15.436ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55192024-11-14T04:49:15.438ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
55202024-11-14T04:49:15.438ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55212024-11-14T04:49:15.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55222024-11-14T04:49:15.439ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
55232024-11-14T04:49:15.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
55242024-11-14T04:49:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
55252024-11-14T04:49:15.440ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
55262024-11-14T04:49:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
55272024-11-14T04:49:15.445ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55282024-11-14T04:49:15.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55292024-11-14T04:49:15.447ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
55302024-11-14T04:49:15.447ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 128.412248ms iteration = 19 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
55312024-11-14T04:49:15.447ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55322024-11-14T04:49:15.450ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
55332024-11-14T04:49:15.450ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55342024-11-14T04:49:15.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55352024-11-14T04:49:15.450ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55362024-11-14T04:49:15.451ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55372024-11-14T04:49:15.451ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
55382024-11-14T04:49:15.451ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55392024-11-14T04:49:15.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55402024-11-14T04:49:15.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
55412024-11-14T04:49:15.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
55422024-11-14T04:49:15.453ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
55432024-11-14T04:49:15.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
55442024-11-14T04:49:15.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
55452024-11-14T04:49:15.458ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
55462024-11-14T04:49:15.458ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
55472024-11-14T04:49:15.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
55482024-11-14T04:49:15.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
55492024-11-14T04:49:15.458ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
55502024-11-14T04:49:15.458ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
55512024-11-14T04:49:15.458ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
55522024-11-14T04:49:15.458ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55532024-11-14T04:49:15.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55542024-11-14T04:49:15.459ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 140.399316ms iteration = 3 status = {"nsuccess":5}
55552024-11-14T04:49:15.460ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
55562024-11-14T04:49:15.460ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
55572024-11-14T04:49:15.460ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 140.89149ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
55582024-11-14T04:49:15.462ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 142.149407ms iteration = 7 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
55592024-11-14T04:49:15.462ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55602024-11-14T04:49:15.466ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
55612024-11-14T04:49:15.466ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55622024-11-14T04:49:15.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55632024-11-14T04:49:15.485ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
55642024-11-14T04:49:15.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
55652024-11-14T04:49:15.485ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
55662024-11-14T04:49:15.485ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
55672024-11-14T04:49:15.485ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
55682024-11-14T04:49:15.485ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55692024-11-14T04:49:15.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55702024-11-14T04:49:15.488ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55712024-11-14T04:49:15.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
55722024-11-14T04:49:15.492ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55732024-11-14T04:49:15.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55742024-11-14T04:49:15.496ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
55752024-11-14T04:49:15.496ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 177.053207ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
55762024-11-14T04:49:15.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55772024-11-14T04:49:15.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55782024-11-14T04:49:15.503ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55792024-11-14T04:49:15.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55802024-11-14T04:49:15.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55812024-11-14T04:49:15.507ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55822024-11-14T04:49:15.507ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
55832024-11-14T04:49:15.507ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 188.632513ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
55842024-11-14T04:49:15.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
55852024-11-14T04:49:15.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55862024-11-14T04:49:15.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55872024-11-14T04:49:15.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 191.629059ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
55882024-11-14T04:49:15.517ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
55892024-11-14T04:49:15.517ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55902024-11-14T04:49:15.520ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
55912024-11-14T04:49:15.520ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55922024-11-14T04:49:15.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55932024-11-14T04:49:15.537ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
55942024-11-14T04:49:15.537ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
55952024-11-14T04:49:15.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55962024-11-14T04:49:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55972024-11-14T04:49:15.540ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 220.643309ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
55982024-11-14T04:49:15.553ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
55992024-11-14T04:49:15.553ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
56002024-11-14T04:49:15.553ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56012024-11-14T04:49:15.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56022024-11-14T04:49:15.565ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56032024-11-14T04:49:15.568ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
56042024-11-14T04:49:15.568ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
56052024-11-14T04:49:15.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
56062024-11-14T04:49:15.586ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
56072024-11-14T04:49:15.586ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 267.480526ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
56082024-11-14T04:49:15.604ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
56092024-11-14T04:49:15.604ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56102024-11-14T04:49:15.605ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ffbdaad-d285-4a08-8571-96fa3e15b0f1", "content-length": "26", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} })
56112024-11-14T04:49:15.605ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56122024-11-14T04:49:15.605ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
56132024-11-14T04:49:15.605ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56142024-11-14T04:49:15.605ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f472dc41-898d-4152-907e-cefb4d02134d", "content-length": "26", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} })
56152024-11-14T04:49:15.605ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56162024-11-14T04:49:15.607ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
56172024-11-14T04:49:15.608ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56182024-11-14T04:49:15.646ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 327.667369ms iteration = 9 status = {"physical_disks_added":0}
56192024-11-14T04:49:15.660ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
56202024-11-14T04:49:15.695ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56212024-11-14T04:49:15.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56222024-11-14T04:49:15.699ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
56232024-11-14T04:49:15.718ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
56242024-11-14T04:49:15.718ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56252024-11-14T04:49:15.718ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "270235d9-fc36-4744-a51c-3a913ad47e42", "content-length": "26", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} })
56262024-11-14T04:49:15.718ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56272024-11-14T04:49:15.721ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56282024-11-14T04:49:15.753ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
56292024-11-14T04:49:15.753ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
56302024-11-14T04:49:15.754ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
56312024-11-14T04:49:15.754ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "816a5ce7-504a-46b3-bf03-75d71881bf85", "content-length": "691", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} })
56322024-11-14T04:49:15.754ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
56332024-11-14T04:49:15.754ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
56342024-11-14T04:49:15.754ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
56352024-11-14T04:49:15.754ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed61eb89-904d-405b-b1e3-ef4ff91f1bea", "content-length": "372", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} })
56362024-11-14T04:49:15.754ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56372024-11-14T04:49:15.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56382024-11-14T04:49:15.755ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 436.360007ms iteration = 7 status = {}
56392024-11-14T04:49:15.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56402024-11-14T04:49:15.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56412024-11-14T04:49:15.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
56422024-11-14T04:49:15.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56432024-11-14T04:49:15.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56442024-11-14T04:49:15.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
56452024-11-14T04:49:15.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56462024-11-14T04:49:15.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56472024-11-14T04:49:15.761ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56482024-11-14T04:49:15.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56492024-11-14T04:49:15.762ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
56502024-11-14T04:49:15.762ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
56512024-11-14T04:49:15.762ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
56522024-11-14T04:49:15.762ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 443.003799ms iteration = 5 status = {"generation":2}
56532024-11-14T04:49:15.762ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
56542024-11-14T04:49:15.762ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
56552024-11-14T04:49:15.762ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
56562024-11-14T04:49:15.762ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 443.415801ms iteration = 5 status = {"generation":1}
56572024-11-14T04:49:15.763ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56582024-11-14T04:49:15.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
56592024-11-14T04:49:15.767ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56602024-11-14T04:49:15.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56612024-11-14T04:49:15.768ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 449.769005ms iteration = 7 status = 0
56622024-11-14T04:49:15.784ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 465.342706ms iteration = 7 status = {}
56632024-11-14T04:49:15.792ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
56642024-11-14T04:49:15.792ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 473.08631ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
56652024-11-14T04:49:15.803ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
56662024-11-14T04:49:15.803ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
56672024-11-14T04:49:15.803ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "46c25bf8-a06c-4b39-a216-0cff2918e813", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} })
56682024-11-14T04:49:15.803ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56692024-11-14T04:49:15.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56702024-11-14T04:49:15.805ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56712024-11-14T04:49:15.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56722024-11-14T04:49:15.809ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56732024-11-14T04:49:15.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56742024-11-14T04:49:15.812ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
56752024-11-14T04:49:15.812ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
56762024-11-14T04:49:15.812ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "11df84d1-b5e7-4437-b0e1-8adb1caffc45", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} })
56772024-11-14T04:49:15.814ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 495.122188ms iteration = 8 status = {}
56782024-11-14T04:49:15.829ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
56792024-11-14T04:49:15.830ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3ea7a57d-3502-40d8-a91c-a55c92dcf046", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56802024-11-14T04:49:15.830ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56812024-11-14T04:49:15.831ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
56822024-11-14T04:49:15.831ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "622aaa7e-35d0-47da-9904-96c893991e37", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56832024-11-14T04:49:15.831ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56842024-11-14T04:49:15.832ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "77cdd820-10a1-4887-85f0-dc5ddde18c72", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56852024-11-14T04:49:15.832ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56862024-11-14T04:49:15.839ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "bb872f5b-75f3-4e8b-97e0-28908782368c", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56872024-11-14T04:49:15.839ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56882024-11-14T04:49:15.841ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56892024-11-14T04:49:15.841ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 521.79609ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
56902024-11-14T04:49:15.842ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a0829a3c-e922-455b-b64d-08fb990b5a87", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56912024-11-14T04:49:15.842ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56922024-11-14T04:49:15.844ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
56932024-11-14T04:49:15.844ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56942024-11-14T04:49:15.844ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 525.328382ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
56952024-11-14T04:49:15.844ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc78b1c8-73a2-4064-9711-d1861c72d558", "content-length": "1", "date": "Thu, 14 Nov 2024 04:49:14 GMT"} })
56962024-11-14T04:49:15.846ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
56972024-11-14T04:49:15.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
56982024-11-14T04:49:15.848ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 529.356608ms iteration = 7 status = null
56992024-11-14T04:49:15.889ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57002024-11-14T04:49:15.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57012024-11-14T04:49:15.891ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
57022024-11-14T04:49:15.891ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57032024-11-14T04:49:15.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57042024-11-14T04:49:15.901ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
57052024-11-14T04:49:15.901ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "aec201ef-1e43-41b9-baec-e446f601b69d", "content-length": "134", "date": "Thu, 14 Nov 2024 04:49:14 GMT"} })
57062024-11-14T04:49:15.901ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
57072024-11-14T04:49:15.901ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
57082024-11-14T04:49:15.901ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
57092024-11-14T04:49:15.901ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
57102024-11-14T04:49:15.901ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57112024-11-14T04:49:15.901ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57122024-11-14T04:49:15.901ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57132024-11-14T04:49:15.901ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
57142024-11-14T04:49:15.902ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57152024-11-14T04:49:15.902ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
57162024-11-14T04:49:15.902ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
57172024-11-14T04:49:15.902ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57182024-11-14T04:49:15.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57192024-11-14T04:49:15.904ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
57202024-11-14T04:49:15.904ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e4ed7b8-d4db-4149-b10e-74564f1b2657", "content-length": "2", "date": "Thu, 14 Nov 2024 04:49:14 GMT"} })
57212024-11-14T04:49:15.904ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
57222024-11-14T04:49:15.904ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1ff2df9-02da-4991-86fc-153e303e58ed", "content-length": "51", "date": "Thu, 14 Nov 2024 04:49:14 GMT"} })
57232024-11-14T04:49:15.904ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
57242024-11-14T04:49:15.904ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
57252024-11-14T04:49:16.003ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "db27ed43-ca81-41a5-b04e-c91519619697", "date": "Thu, 14 Nov 2024 04:49:15 GMT"} })
57262024-11-14T04:49:16.004ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
57272024-11-14T04:49:16.004ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
57282024-11-14T04:49:16.004ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e251b9f-6255-4b05-9788-a0643c4f4c7e", "content-length": "682", "date": "Thu, 14 Nov 2024 04:49:16 GMT"} })
57292024-11-14T04:49:16.004ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57302024-11-14T04:49:16.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57312024-11-14T04:49:16.009ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57322024-11-14T04:49:16.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57332024-11-14T04:49:16.013ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57342024-11-14T04:49:16.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57352024-11-14T04:49:16.015ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57362024-11-14T04:49:16.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57372024-11-14T04:49:16.022ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 702.828212ms iteration = 7 status = {}
57382024-11-14T04:49:19.630ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:47901
57392024-11-14T04:49:19.630ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:47901 req_id = e81b7680-e639-4a64-9044-d1fb5e5c4906 roles = RoleSet { roles: {} } uri = /metrics/producers
57402024-11-14T04:49:19.630ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:47901 req_id = e81b7680-e639-4a64-9044-d1fb5e5c4906 resource = Database result = Ok(()) uri = /metrics/producers
57412024-11-14T04:49:19.668ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = e5586e29-e301-4b24-9a2c-39545da75716
57422024-11-14T04:49:19.669ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38817 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:47901 req_id = e81b7680-e639-4a64-9044-d1fb5e5c4906 response_code = 201 uri = /metrics/producers
57432024-11-14T04:49:20.262ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1121 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = b11b5a7b-46e0-4fd4-8d49-b50756e329d0 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
57442024-11-14T04:49:25.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = d32edf4f-50d9-4af7-ad05-2377a94206f8 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
57452024-11-14T04:49:25.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = d32edf4f-50d9-4af7-ad05-2377a94206f8 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
57462024-11-14T04:49:25.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3322 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = d32edf4f-50d9-4af7-ad05-2377a94206f8 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
57472024-11-14T04:49:25.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = e0565209-e372-4694-a33e-c91f8833eaf1 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
57482024-11-14T04:49:25.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = e0565209-e372-4694-a33e-c91f8833eaf1 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
57492024-11-14T04:49:25.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2201 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47769 req_id = e0565209-e372-4694-a33e-c91f8833eaf1 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
57502024-11-14T04:49:25.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
57512024-11-14T04:49:25.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
57522024-11-14T04:49:25.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57532024-11-14T04:49:25.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57542024-11-14T04:49:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57552024-11-14T04:49:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57562024-11-14T04:49:25.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57572024-11-14T04:49:25.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57582024-11-14T04:49:25.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57592024-11-14T04:49:25.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.606184ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
57602024-11-14T04:49:25.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
57612024-11-14T04:49:25.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57622024-11-14T04:49:25.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57632024-11-14T04:49:25.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57642024-11-14T04:49:25.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57652024-11-14T04:49:25.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57662024-11-14T04:49:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
57672024-11-14T04:49:25.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57682024-11-14T04:49:25.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57692024-11-14T04:49:25.340ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
57702024-11-14T04:49:25.340ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.02758ms iteration = 20 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
57712024-11-14T04:49:30.606ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:43008
57722024-11-14T04:49:30.606ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:43008 req_id = d4cd643b-484e-4d90-a5ec-81f1a20f2ebc roles = RoleSet { roles: {} } uri = /metrics/producers
57732024-11-14T04:49:30.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:43008 req_id = d4cd643b-484e-4d90-a5ec-81f1a20f2ebc resource = Database result = Ok(()) uri = /metrics/producers
57742024-11-14T04:49:30.643ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = f4c8f330-91d6-4976-b637-c72f900d8c87
57752024-11-14T04:49:30.643ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36202 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:43008 req_id = d4cd643b-484e-4d90-a5ec-81f1a20f2ebc response_code = 201 uri = /metrics/producers
57762024-11-14T04:49:32.558ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:59169
57772024-11-14T04:49:32.558ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:59169 req_id = 6cf91b6d-cf49-4352-ba31-096e1c2e2cf5 roles = RoleSet { roles: {} } uri = /metrics/producers
57782024-11-14T04:49:32.559ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:59169 req_id = 6cf91b6d-cf49-4352-ba31-096e1c2e2cf5 resource = Database result = Ok(()) uri = /metrics/producers
57792024-11-14T04:49:32.570ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 18d4ee8d-1d67-44aa-96f9-89dfcadba34d
57802024-11-14T04:49:32.570ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11600 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:59169 req_id = 6cf91b6d-cf49-4352-ba31-096e1c2e2cf5 response_code = 201 uri = /metrics/producers
57812024-11-14T04:49:35.262ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1239 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 19c45faf-17ed-4983-bcaa-ac90f00f2b21 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
57822024-11-14T04:49:35.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
57832024-11-14T04:49:35.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57842024-11-14T04:49:35.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
57852024-11-14T04:49:35.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57862024-11-14T04:49:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57872024-11-14T04:49:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57882024-11-14T04:49:35.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57892024-11-14T04:49:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
57902024-11-14T04:49:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57912024-11-14T04:49:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57922024-11-14T04:49:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57932024-11-14T04:49:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57942024-11-14T04:49:35.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57952024-11-14T04:49:35.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
57962024-11-14T04:49:35.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57972024-11-14T04:49:35.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57982024-11-14T04:49:35.341ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
57992024-11-14T04:49:35.341ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.699798ms iteration = 21 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
58002024-11-14T04:49:35.348ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58012024-11-14T04:49:35.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58022024-11-14T04:49:35.350ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.06952ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58032024-11-14T04:49:44.372ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:37735
58042024-11-14T04:49:44.372ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:37735 req_id = ee1fe452-02b3-405c-978e-c99aa246b529 roles = RoleSet { roles: {} } uri = /metrics/producers
58052024-11-14T04:49:44.372ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:37735 req_id = ee1fe452-02b3-405c-978e-c99aa246b529 resource = Database result = Ok(()) uri = /metrics/producers
58062024-11-14T04:49:44.407ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = bc37b1eb-3677-438e-a276-b6d3e6fe3f8e
58072024-11-14T04:49:44.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35457 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:37735 req_id = ee1fe452-02b3-405c-978e-c99aa246b529 response_code = 201 uri = /metrics/producers
58082024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
58092024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
58102024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58112024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58122024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
58132024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
58142024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
58152024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58162024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
58172024-11-14T04:49:45.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
58182024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
58192024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
58202024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58212024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58222024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
58232024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
58242024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
58252024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
58262024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
58272024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58282024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58292024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
58302024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58312024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
58322024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
58332024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
58342024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58352024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
58362024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58372024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
58382024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
58392024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
58402024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
58412024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
58422024-11-14T04:49:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
58432024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58442024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
58452024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
58462024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
58472024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
58482024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
58492024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58502024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58512024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58522024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58532024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
58542024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
58552024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
58562024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58572024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
58582024-11-14T04:49:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58592024-11-14T04:49:45.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58602024-11-14T04:49:45.323ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1164, tv_nsec: 96056700 } })
58612024-11-14T04:49:45.323ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
58622024-11-14T04:49:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58632024-11-14T04:49:45.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58642024-11-14T04:49:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58652024-11-14T04:49:45.326ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.212969ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
58662024-11-14T04:49:45.327ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58672024-11-14T04:49:45.327ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1164, tv_nsec: 100889720 } })
58682024-11-14T04:49:45.328ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
58692024-11-14T04:49:45.328ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58702024-11-14T04:49:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
58712024-11-14T04:49:45.329ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58722024-11-14T04:49:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
58732024-11-14T04:49:45.331ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58742024-11-14T04:49:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
58752024-11-14T04:49:45.331ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58762024-11-14T04:49:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
58772024-11-14T04:49:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58782024-11-14T04:49:45.333ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58792024-11-14T04:49:45.334ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 14.884372ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
58802024-11-14T04:49:45.334ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 15.101777ms iteration = 8 status = {"errors":[],"records_set_to_done":[]}
58812024-11-14T04:49:45.335ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58822024-11-14T04:49:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58832024-11-14T04:49:45.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58842024-11-14T04:49:45.336ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58852024-11-14T04:49:45.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58862024-11-14T04:49:45.338ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58872024-11-14T04:49:45.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58882024-11-14T04:49:45.339ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58892024-11-14T04:49:45.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58902024-11-14T04:49:45.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58912024-11-14T04:49:45.340ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 20.870564ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
58922024-11-14T04:49:45.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
58932024-11-14T04:49:45.341ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58942024-11-14T04:49:45.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58952024-11-14T04:49:45.344ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58962024-11-14T04:49:45.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58972024-11-14T04:49:45.347ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58982024-11-14T04:49:45.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58992024-11-14T04:49:45.348ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1164, tv_nsec: 121532507 } })
59002024-11-14T04:49:45.349ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
59012024-11-14T04:49:45.382ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59022024-11-14T04:49:45.383ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
59032024-11-14T04:49:45.383ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59042024-11-14T04:49:45.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59052024-11-14T04:49:45.392ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59062024-11-14T04:49:45.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59072024-11-14T04:49:45.394ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59082024-11-14T04:49:45.395ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59092024-11-14T04:49:45.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59102024-11-14T04:49:45.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59112024-11-14T04:49:45.399ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59122024-11-14T04:49:45.401ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59132024-11-14T04:49:45.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59142024-11-14T04:49:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59152024-11-14T04:49:45.402ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59162024-11-14T04:49:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59172024-11-14T04:49:45.405ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59182024-11-14T04:49:45.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59192024-11-14T04:49:45.407ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59202024-11-14T04:49:45.407ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59212024-11-14T04:49:45.407ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
59222024-11-14T04:49:45.408ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 88.304867ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
59232024-11-14T04:49:45.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 90.249904ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
59242024-11-14T04:49:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59252024-11-14T04:49:45.411ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59262024-11-14T04:49:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
59272024-11-14T04:49:45.411ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59282024-11-14T04:49:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59292024-11-14T04:49:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59302024-11-14T04:49:45.413ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 93.960711ms iteration = 8 status = 0
59312024-11-14T04:49:45.416ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
59322024-11-14T04:49:45.416ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59332024-11-14T04:49:45.416ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d08a6055-47b4-4040-b5a8-23a6086a1bd6", "content-length": "26", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
59342024-11-14T04:49:45.416ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59352024-11-14T04:49:45.419ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
59362024-11-14T04:49:45.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59372024-11-14T04:49:45.426ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
59382024-11-14T04:49:45.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59392024-11-14T04:49:45.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59402024-11-14T04:49:45.445ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59412024-11-14T04:49:45.445ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
59422024-11-14T04:49:45.445ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59432024-11-14T04:49:45.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59442024-11-14T04:49:45.445ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
59452024-11-14T04:49:45.445ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 126.214055ms iteration = 22 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
59462024-11-14T04:49:45.451ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59472024-11-14T04:49:45.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59482024-11-14T04:49:45.451ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
59492024-11-14T04:49:45.451ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59502024-11-14T04:49:45.452ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d68be0fc-f124-4bec-9498-3ca624ba750b", "content-length": "26", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
59512024-11-14T04:49:45.452ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59522024-11-14T04:49:45.453ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
59532024-11-14T04:49:45.453ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59542024-11-14T04:49:45.453ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b3957c4-247d-4ddd-b131-0c572286930c", "content-length": "26", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
59552024-11-14T04:49:45.453ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59562024-11-14T04:49:45.454ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
59572024-11-14T04:49:45.454ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 134.848345ms iteration = 8 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
59582024-11-14T04:49:45.455ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
59592024-11-14T04:49:45.462ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59602024-11-14T04:49:45.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59612024-11-14T04:49:45.492ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59622024-11-14T04:49:45.492ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
59632024-11-14T04:49:45.492ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 172.927288ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
59642024-11-14T04:49:45.496ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 176.186372ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
59652024-11-14T04:49:45.544ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 225.176691ms iteration = 10 status = {"physical_disks_added":0}
59662024-11-14T04:49:45.547ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
59672024-11-14T04:49:45.547ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "60071867-ac0a-4680-acdf-3bab3c03403a", "content-length": "1", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
59682024-11-14T04:49:45.549ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
59692024-11-14T04:49:45.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
59702024-11-14T04:49:45.550ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 231.044881ms iteration = 8 status = null
59712024-11-14T04:49:45.593ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59722024-11-14T04:49:45.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59732024-11-14T04:49:45.595ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59742024-11-14T04:49:45.597ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
59752024-11-14T04:49:45.597ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59762024-11-14T04:49:45.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59772024-11-14T04:49:45.662ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 342.979243ms iteration = 8 status = {}
59782024-11-14T04:49:45.672ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
59792024-11-14T04:49:45.672ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
59802024-11-14T04:49:45.673ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4353f003-c6f4-49e9-b2c0-37c4415cf8e3", "content-length": "372", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
59812024-11-14T04:49:45.673ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59822024-11-14T04:49:45.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59832024-11-14T04:49:45.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59842024-11-14T04:49:45.676ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
59852024-11-14T04:49:45.676ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59862024-11-14T04:49:45.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59872024-11-14T04:49:45.720ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
59882024-11-14T04:49:45.720ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
59892024-11-14T04:49:45.724ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59902024-11-14T04:49:45.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59912024-11-14T04:49:45.727ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
59922024-11-14T04:49:45.727ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59932024-11-14T04:49:45.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59942024-11-14T04:49:45.732ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
59952024-11-14T04:49:45.783ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3a67c5ee-3b78-4ec7-a8bd-195c56370b8c", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
59962024-11-14T04:49:45.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59972024-11-14T04:49:45.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59982024-11-14T04:49:45.783ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c032076-7086-4e27-a3e8-4bca3e0b20ac", "content-length": "691", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
59992024-11-14T04:49:45.783ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
60002024-11-14T04:49:45.783ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
60012024-11-14T04:49:45.783ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
60022024-11-14T04:49:45.784ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 465.219092ms iteration = 8 status = {}
60032024-11-14T04:49:45.786ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60042024-11-14T04:49:45.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60052024-11-14T04:49:45.789ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60062024-11-14T04:49:45.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60072024-11-14T04:49:45.793ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
60082024-11-14T04:49:45.793ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60092024-11-14T04:49:45.793ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f2eff722-ef83-4839-9bea-63045fce5fa6", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
60102024-11-14T04:49:45.794ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 474.945124ms iteration = 9 status = {}
60112024-11-14T04:49:45.935ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
60122024-11-14T04:49:45.936ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "deb5fdf2-a71a-4e6d-aa36-6f287e7658ea", "content-length": "134", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
60132024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
60142024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
60152024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
60162024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
60172024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60182024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60192024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60202024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
60212024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60222024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
60232024-11-14T04:49:45.936ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
60242024-11-14T04:49:45.936ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60252024-11-14T04:49:45.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60262024-11-14T04:49:45.939ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
60272024-11-14T04:49:45.939ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "923f4b0f-2f27-4f6a-96e2-b6097e00f673", "content-length": "2", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
60282024-11-14T04:49:45.939ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
60292024-11-14T04:49:45.939ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "58084e00-02d6-44a0-be35-acda3385e36c", "content-length": "51", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
60302024-11-14T04:49:45.939ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
60312024-11-14T04:49:45.939ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
60322024-11-14T04:49:46.036ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "09bc636c-2dbd-44d6-8c0f-a31e643d4478", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
60332024-11-14T04:49:46.036ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
60342024-11-14T04:49:46.036ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
60352024-11-14T04:49:46.036ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8ca05c0-0b2b-4ac1-94d2-679fae8c5d9d", "content-length": "682", "date": "Thu, 14 Nov 2024 04:49:45 GMT"} })
60362024-11-14T04:49:46.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60372024-11-14T04:49:46.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60382024-11-14T04:49:46.040ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60392024-11-14T04:49:46.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60402024-11-14T04:49:46.043ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60412024-11-14T04:49:46.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60422024-11-14T04:49:46.045ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60432024-11-14T04:49:46.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60442024-11-14T04:49:46.052ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 732.487141ms iteration = 8 status = {}
60452024-11-14T04:49:49.030ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:54396
60462024-11-14T04:49:49.030ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54396 req_id = 603d56a1-041b-4af8-a3b0-86f3c35de462 roles = RoleSet { roles: {} } uri = /metrics/producers
60472024-11-14T04:49:49.030ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54396 req_id = 603d56a1-041b-4af8-a3b0-86f3c35de462 resource = Database result = Ok(()) uri = /metrics/producers
60482024-11-14T04:49:49.064ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = c073d51c-abaa-48df-abb6-2f053e9c1bec
60492024-11-14T04:49:49.064ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34131 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54396 req_id = 603d56a1-041b-4af8-a3b0-86f3c35de462 response_code = 201 uri = /metrics/producers
60502024-11-14T04:49:50.262ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1191 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = d9ec56b3-2fc1-49db-af4d-017cdbd54443 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
60512024-11-14T04:49:55.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
60522024-11-14T04:49:55.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
60532024-11-14T04:49:55.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60542024-11-14T04:49:55.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60552024-11-14T04:49:55.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60562024-11-14T04:49:55.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60572024-11-14T04:49:55.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60582024-11-14T04:49:55.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60592024-11-14T04:49:55.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60602024-11-14T04:49:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60612024-11-14T04:49:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60622024-11-14T04:49:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60632024-11-14T04:49:55.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60642024-11-14T04:49:55.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60652024-11-14T04:49:55.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60662024-11-14T04:49:55.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
60672024-11-14T04:49:55.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60682024-11-14T04:49:55.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60692024-11-14T04:49:55.344ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
60702024-11-14T04:49:55.344ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.644753ms iteration = 23 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
60712024-11-14T04:49:55.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.737752ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60722024-11-14T04:50:05.262ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1313 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 41feca67-30cc-4e89-8d52-7ae7f058cd71 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
60732024-11-14T04:50:05.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
60742024-11-14T04:50:05.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
60752024-11-14T04:50:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60762024-11-14T04:50:05.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60772024-11-14T04:50:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60782024-11-14T04:50:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60792024-11-14T04:50:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60802024-11-14T04:50:05.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60812024-11-14T04:50:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60822024-11-14T04:50:05.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.174131ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60832024-11-14T04:50:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60842024-11-14T04:50:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60852024-11-14T04:50:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60862024-11-14T04:50:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60872024-11-14T04:50:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60882024-11-14T04:50:05.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60892024-11-14T04:50:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
60902024-11-14T04:50:05.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60912024-11-14T04:50:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60922024-11-14T04:50:05.337ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
60932024-11-14T04:50:05.337ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.191667ms iteration = 24 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
60942024-11-14T04:50:11.612ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:62015
60952024-11-14T04:50:11.612ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:62015 req_id = ff00e43d-aede-48b2-89e5-cda5ab218698 roles = RoleSet { roles: {} } uri = /metrics/producers
60962024-11-14T04:50:11.612ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:62015 req_id = ff00e43d-aede-48b2-89e5-cda5ab218698 resource = Database result = Ok(()) uri = /metrics/producers
60972024-11-14T04:50:11.626ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = b584ea61-acb3-47a3-a173-0d03f6cae9c9
60982024-11-14T04:50:11.626ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 14552 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:62015 req_id = ff00e43d-aede-48b2-89e5-cda5ab218698 response_code = 201 uri = /metrics/producers
60992024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
61002024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
61012024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61022024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
61032024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
61042024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
61052024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
61062024-11-14T04:50:15.319ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:40:15.319454783 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
61072024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
61082024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
61092024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
61102024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
61112024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61122024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
61132024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
61142024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
61152024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
61162024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61172024-11-14T04:50:15.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
61182024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
61192024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61202024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
61212024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
61222024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
61232024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61242024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
61252024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
61262024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
61272024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
61282024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
61292024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
61302024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
61312024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61322024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
61332024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
61342024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
61352024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
61362024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
61372024-11-14T04:50:15.319ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
61382024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
61392024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.461\u{b5}s iteration = 6 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
61402024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
61412024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61422024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61432024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
61442024-11-14T04:50:15.319ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
61452024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
61462024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
61472024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
61482024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61492024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
61502024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
61512024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
61522024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
61532024-11-14T04:50:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
61542024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
61552024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61562024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
61572024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
61582024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
61592024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
61602024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
61612024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
61622024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61632024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
61642024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
61652024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
61662024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
61672024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61682024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
61692024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
61702024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
61712024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
61722024-11-14T04:50:15.320ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1194, tv_nsec: 93434399 } })
61732024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
61742024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61752024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61762024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61772024-11-14T04:50:15.320ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1194, tv_nsec: 93500161 } })
61782024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61792024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
61802024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
61812024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
61822024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61832024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
61842024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
61852024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
61862024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
61872024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
61882024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.270647ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
61892024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
61902024-11-14T04:50:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.427512ms iteration = 6 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
61912024-11-14T04:50:15.321ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61922024-11-14T04:50:15.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61932024-11-14T04:50:15.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61942024-11-14T04:50:15.323ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61952024-11-14T04:50:15.323ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.009076ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
61962024-11-14T04:50:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
61972024-11-14T04:50:15.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61982024-11-14T04:50:15.324ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:40:15.319454783 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
61992024-11-14T04:50:15.324ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.027395ms iteration = 5 status = {"expiration":"2024-11-14T04:40:15.319454783Z","pruned":[]}
62002024-11-14T04:50:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62012024-11-14T04:50:15.325ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1194, tv_nsec: 98647121 } })
62022024-11-14T04:50:15.325ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
62032024-11-14T04:50:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
62042024-11-14T04:50:15.326ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62052024-11-14T04:50:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
62062024-11-14T04:50:15.330ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1194, tv_nsec: 103558063 } })
62072024-11-14T04:50:15.330ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
62082024-11-14T04:50:15.330ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62092024-11-14T04:50:15.331ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62102024-11-14T04:50:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
62112024-11-14T04:50:15.333ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
62122024-11-14T04:50:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
62132024-11-14T04:50:15.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
62142024-11-14T04:50:15.334ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62152024-11-14T04:50:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
62162024-11-14T04:50:15.337ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62172024-11-14T04:50:15.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62182024-11-14T04:50:15.339ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62192024-11-14T04:50:15.340ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
62202024-11-14T04:50:15.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
62212024-11-14T04:50:15.341ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.527944ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
62222024-11-14T04:50:15.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
62232024-11-14T04:50:15.342ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
62242024-11-14T04:50:15.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
62252024-11-14T04:50:15.343ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 23.262914ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
62262024-11-14T04:50:15.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62272024-11-14T04:50:15.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62282024-11-14T04:50:15.349ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62292024-11-14T04:50:15.352ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62302024-11-14T04:50:15.352ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62312024-11-14T04:50:15.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
62322024-11-14T04:50:15.352ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
62332024-11-14T04:50:15.352ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62342024-11-14T04:50:15.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
62352024-11-14T04:50:15.354ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
62362024-11-14T04:50:15.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62372024-11-14T04:50:15.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
62382024-11-14T04:50:15.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62392024-11-14T04:50:15.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62402024-11-14T04:50:15.388ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
62412024-11-14T04:50:15.388ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
62422024-11-14T04:50:15.388ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62432024-11-14T04:50:15.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
62442024-11-14T04:50:15.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 90.568632ms iteration = 9 status = {"errors":[],"records_set_to_done":[]}
62452024-11-14T04:50:15.414ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62462024-11-14T04:50:15.414ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62472024-11-14T04:50:15.414ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62482024-11-14T04:50:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62492024-11-14T04:50:15.417ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62502024-11-14T04:50:15.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
62512024-11-14T04:50:15.418ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62522024-11-14T04:50:15.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62532024-11-14T04:50:15.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
62542024-11-14T04:50:15.421ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62552024-11-14T04:50:15.421ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62562024-11-14T04:50:15.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62572024-11-14T04:50:15.424ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1194, tv_nsec: 197690078 } })
62582024-11-14T04:50:15.424ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
62592024-11-14T04:50:15.425ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62602024-11-14T04:50:15.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62612024-11-14T04:50:15.427ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
62622024-11-14T04:50:15.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
62632024-11-14T04:50:15.429ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
62642024-11-14T04:50:15.429ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
62652024-11-14T04:50:15.429ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
62662024-11-14T04:50:15.429ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62672024-11-14T04:50:15.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62682024-11-14T04:50:15.430ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
62692024-11-14T04:50:15.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
62702024-11-14T04:50:15.430ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
62712024-11-14T04:50:15.430ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 111.023405ms iteration = 25 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
62722024-11-14T04:50:15.431ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62732024-11-14T04:50:15.432ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62742024-11-14T04:50:15.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
62752024-11-14T04:50:15.435ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62762024-11-14T04:50:15.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
62772024-11-14T04:50:15.436ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
62782024-11-14T04:50:15.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62792024-11-14T04:50:15.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62802024-11-14T04:50:15.438ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62812024-11-14T04:50:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62822024-11-14T04:50:15.440ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62832024-11-14T04:50:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62842024-11-14T04:50:15.443ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62852024-11-14T04:50:15.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
62862024-11-14T04:50:15.444ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
62872024-11-14T04:50:15.444ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 124.99662ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
62882024-11-14T04:50:15.444ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62892024-11-14T04:50:15.448ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 127.576295ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
62902024-11-14T04:50:15.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
62912024-11-14T04:50:15.449ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62922024-11-14T04:50:15.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
62932024-11-14T04:50:15.460ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62942024-11-14T04:50:15.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
62952024-11-14T04:50:15.462ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
62962024-11-14T04:50:15.462ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
62972024-11-14T04:50:15.462ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 142.825016ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
62982024-11-14T04:50:15.463ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
62992024-11-14T04:50:15.463ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
63002024-11-14T04:50:15.463ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 144.156195ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
63012024-11-14T04:50:15.464ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63022024-11-14T04:50:15.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63032024-11-14T04:50:15.465ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63042024-11-14T04:50:15.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63052024-11-14T04:50:15.467ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 147.233254ms iteration = 9 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
63062024-11-14T04:50:15.467ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
63072024-11-14T04:50:15.467ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63082024-11-14T04:50:15.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
63092024-11-14T04:50:15.471ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63102024-11-14T04:50:15.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63112024-11-14T04:50:15.474ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 155.536784ms iteration = 9 status = 0
63122024-11-14T04:50:15.481ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
63132024-11-14T04:50:15.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
63142024-11-14T04:50:15.500ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 180.563228ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
63152024-11-14T04:50:15.500ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
63162024-11-14T04:50:15.500ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
63172024-11-14T04:50:15.500ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
63182024-11-14T04:50:15.500ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63192024-11-14T04:50:15.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63202024-11-14T04:50:15.501ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 182.163796ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
63212024-11-14T04:50:15.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63222024-11-14T04:50:15.503ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
63232024-11-14T04:50:15.503ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
63242024-11-14T04:50:15.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
63252024-11-14T04:50:15.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
63262024-11-14T04:50:15.506ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
63272024-11-14T04:50:15.506ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
63282024-11-14T04:50:15.506ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 187.091407ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
63292024-11-14T04:50:15.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
63302024-11-14T04:50:15.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63312024-11-14T04:50:15.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63322024-11-14T04:50:15.510ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
63332024-11-14T04:50:15.510ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
63342024-11-14T04:50:15.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63352024-11-14T04:50:15.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63362024-11-14T04:50:15.515ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
63372024-11-14T04:50:15.515ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
63382024-11-14T04:50:15.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
63392024-11-14T04:50:15.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
63402024-11-14T04:50:15.518ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 198.8933ms iteration = 11 status = {"physical_disks_added":0}
63412024-11-14T04:50:15.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63422024-11-14T04:50:15.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
63432024-11-14T04:50:15.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
63442024-11-14T04:50:15.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
63452024-11-14T04:50:15.527ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
63462024-11-14T04:50:15.527ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
63472024-11-14T04:50:15.527ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "359e923d-ddbd-4ef7-a3cb-58a757dad61d", "content-length": "26", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
63482024-11-14T04:50:15.527ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
63492024-11-14T04:50:15.530ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
63502024-11-14T04:50:15.533ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
63512024-11-14T04:50:15.533ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
63522024-11-14T04:50:15.533ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
63532024-11-14T04:50:15.533ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 213.702409ms iteration = 6 status = {"generation":1}
63542024-11-14T04:50:15.547ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63552024-11-14T04:50:15.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63562024-11-14T04:50:15.548ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63572024-11-14T04:50:15.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
63582024-11-14T04:50:15.552ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63592024-11-14T04:50:15.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63602024-11-14T04:50:15.565ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
63612024-11-14T04:50:15.565ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
63622024-11-14T04:50:15.565ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce9349bf-4cff-45b9-92f8-b7e25163fce9", "content-length": "26", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
63632024-11-14T04:50:15.565ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
63642024-11-14T04:50:15.567ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
63652024-11-14T04:50:15.572ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
63662024-11-14T04:50:15.572ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
63672024-11-14T04:50:15.572ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd1f0a52-ab25-483a-8593-213236131452", "content-length": "26", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
63682024-11-14T04:50:15.572ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
63692024-11-14T04:50:15.574ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
63702024-11-14T04:50:15.597ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
63712024-11-14T04:50:15.609ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
63722024-11-14T04:50:15.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
63732024-11-14T04:50:15.632ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
63742024-11-14T04:50:15.632ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 313.242111ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
63752024-11-14T04:50:15.641ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
63762024-11-14T04:50:15.691ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63772024-11-14T04:50:15.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63782024-11-14T04:50:15.719ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 399.503539ms iteration = 9 status = {}
63792024-11-14T04:50:15.756ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
63802024-11-14T04:50:15.756ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c313ee1-1606-4a86-83a8-8f76c33625ba", "content-length": "1", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
63812024-11-14T04:50:15.758ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
63822024-11-14T04:50:15.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
63832024-11-14T04:50:15.796ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
63842024-11-14T04:50:15.797ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "595e9bae-0d7a-4a6f-9765-fb9e70f2a653", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
63852024-11-14T04:50:15.797ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
63862024-11-14T04:50:15.798ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "52359b32-27a2-4b05-9d0d-69b7b0302f4a", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
63872024-11-14T04:50:15.798ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
63882024-11-14T04:50:15.799ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
63892024-11-14T04:50:15.799ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
63902024-11-14T04:50:15.799ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "05bec9fb-e58c-4a45-8580-c9b5b5384b27", "content-length": "372", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
63912024-11-14T04:50:15.800ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63922024-11-14T04:50:15.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63932024-11-14T04:50:15.801ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "43a6959d-628a-441e-9c76-7c44935e57f5", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
63942024-11-14T04:50:15.801ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
63952024-11-14T04:50:15.802ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63962024-11-14T04:50:15.802ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63972024-11-14T04:50:15.802ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
63982024-11-14T04:50:15.802ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
63992024-11-14T04:50:15.802ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
64002024-11-14T04:50:15.802ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 483.416408ms iteration = 6 status = {"generation":2}
64012024-11-14T04:50:15.803ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
64022024-11-14T04:50:15.803ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 483.710147ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
64032024-11-14T04:50:15.804ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 484.597412ms iteration = 9 status = null
64042024-11-14T04:50:15.805ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64052024-11-14T04:50:15.805ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64062024-11-14T04:50:15.805ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
64072024-11-14T04:50:15.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64082024-11-14T04:50:15.805ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64092024-11-14T04:50:15.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
64102024-11-14T04:50:15.808ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64112024-11-14T04:50:15.811ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64122024-11-14T04:50:15.811ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64132024-11-14T04:50:15.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64142024-11-14T04:50:15.814ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
64152024-11-14T04:50:15.815ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "fd111729-84c0-4bb3-86a9-3cdc4deaf989", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
64162024-11-14T04:50:15.815ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
64172024-11-14T04:50:15.817ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c40b5fe9-c91c-4699-82ce-bf617e71a274", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
64182024-11-14T04:50:15.817ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
64192024-11-14T04:50:15.819ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
64202024-11-14T04:50:15.819ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 499.864675ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
64212024-11-14T04:50:15.820ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
64222024-11-14T04:50:15.820ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
64232024-11-14T04:50:15.820ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3f51e5d0-d238-4df2-979f-424b4270f274", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
64242024-11-14T04:50:15.820ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64252024-11-14T04:50:15.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64262024-11-14T04:50:15.823ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64272024-11-14T04:50:15.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64282024-11-14T04:50:15.826ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64292024-11-14T04:50:15.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64302024-11-14T04:50:15.831ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
64312024-11-14T04:50:15.831ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
64322024-11-14T04:50:15.831ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c2156455-8de2-4464-8043-55e57e57464f", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
64332024-11-14T04:50:15.834ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 514.735745ms iteration = 10 status = {}
64342024-11-14T04:50:15.851ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
64352024-11-14T04:50:15.851ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 532.338175ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
64362024-11-14T04:50:15.887ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
64372024-11-14T04:50:15.888ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "909fcdad-3eff-426f-96de-afaa87c886cc", "content-length": "691", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
64382024-11-14T04:50:15.888ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
64392024-11-14T04:50:15.888ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
64402024-11-14T04:50:15.888ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
64412024-11-14T04:50:15.890ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 570.934673ms iteration = 9 status = {}
64422024-11-14T04:50:15.903ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64432024-11-14T04:50:15.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64442024-11-14T04:50:15.906ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
64452024-11-14T04:50:15.906ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64462024-11-14T04:50:15.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64472024-11-14T04:50:16.123ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
64482024-11-14T04:50:16.123ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf9947d-0ebc-4bbe-bb71-da603690e987", "content-length": "134", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
64492024-11-14T04:50:16.123ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
64502024-11-14T04:50:16.123ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
64512024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
64522024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
64532024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64542024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64552024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64562024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
64572024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64582024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
64592024-11-14T04:50:16.124ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
64602024-11-14T04:50:16.124ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64612024-11-14T04:50:16.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64622024-11-14T04:50:16.127ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
64632024-11-14T04:50:16.127ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "fac64974-b34b-42a1-826e-03708a683742", "content-length": "2", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
64642024-11-14T04:50:16.127ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
64652024-11-14T04:50:16.127ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "2214e269-6b3b-48de-ae75-73c602bca272", "content-length": "51", "date": "Thu, 14 Nov 2024 04:50:15 GMT"} })
64662024-11-14T04:50:16.127ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
64672024-11-14T04:50:16.127ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
64682024-11-14T04:50:16.225ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "fee4af12-d9f6-4179-99dc-1dfd66190f9e", "date": "Thu, 14 Nov 2024 04:50:16 GMT"} })
64692024-11-14T04:50:16.225ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
64702024-11-14T04:50:16.225ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
64712024-11-14T04:50:16.313ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "18424d71-3ce2-4ce0-a563-497143e513d8", "content-length": "682", "date": "Thu, 14 Nov 2024 04:50:16 GMT"} })
64722024-11-14T04:50:16.313ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64732024-11-14T04:50:16.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64742024-11-14T04:50:16.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64752024-11-14T04:50:16.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64762024-11-14T04:50:16.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64772024-11-14T04:50:16.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64782024-11-14T04:50:16.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64792024-11-14T04:50:16.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64802024-11-14T04:50:16.328ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.009017379s iteration = 9 status = {}
64812024-11-14T04:50:20.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1128 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = b2ee26cc-8637-437c-ad5d-acedbc783fc3 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
64822024-11-14T04:50:25.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
64832024-11-14T04:50:25.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
64842024-11-14T04:50:25.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64852024-11-14T04:50:25.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64862024-11-14T04:50:25.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64872024-11-14T04:50:25.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64882024-11-14T04:50:25.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64892024-11-14T04:50:25.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64902024-11-14T04:50:25.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64912024-11-14T04:50:25.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.325855ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
64922024-11-14T04:50:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
64932024-11-14T04:50:25.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64942024-11-14T04:50:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64952024-11-14T04:50:25.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64962024-11-14T04:50:25.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64972024-11-14T04:50:25.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64982024-11-14T04:50:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
64992024-11-14T04:50:25.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65002024-11-14T04:50:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65012024-11-14T04:50:25.340ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
65022024-11-14T04:50:25.340ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.166973ms iteration = 26 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
65032024-11-14T04:50:35.261ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1128 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = a9885b8e-a4eb-473b-bfec-f7f060a43d05 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
65042024-11-14T04:50:35.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
65052024-11-14T04:50:35.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
65062024-11-14T04:50:35.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65072024-11-14T04:50:35.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65082024-11-14T04:50:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65092024-11-14T04:50:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65102024-11-14T04:50:35.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65112024-11-14T04:50:35.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65122024-11-14T04:50:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65132024-11-14T04:50:35.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.132789ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65142024-11-14T04:50:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65152024-11-14T04:50:35.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65162024-11-14T04:50:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65172024-11-14T04:50:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65182024-11-14T04:50:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65192024-11-14T04:50:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65202024-11-14T04:50:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
65212024-11-14T04:50:35.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65222024-11-14T04:50:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65232024-11-14T04:50:35.336ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
65242024-11-14T04:50:35.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.469506ms iteration = 27 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
65252024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
65262024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
65272024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65282024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
65292024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
65302024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65312024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65322024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65332024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
65342024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
65352024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65362024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65372024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
65382024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
65392024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
65402024-11-14T04:50:45.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
65412024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
65422024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
65432024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65442024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
65452024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
65462024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
65472024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
65482024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
65492024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
65502024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
65512024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
65522024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
65532024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
65542024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
65552024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
65562024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
65572024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65582024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
65592024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
65602024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
65612024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
65622024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65632024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65642024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65652024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65662024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
65672024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
65682024-11-14T04:50:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65692024-11-14T04:50:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
65702024-11-14T04:50:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65712024-11-14T04:50:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
65722024-11-14T04:50:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
65732024-11-14T04:50:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
65742024-11-14T04:50:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
65752024-11-14T04:50:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
65762024-11-14T04:50:45.322ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.596575ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
65772024-11-14T04:50:45.322ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65782024-11-14T04:50:45.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65792024-11-14T04:50:45.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65802024-11-14T04:50:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
65812024-11-14T04:50:45.325ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65822024-11-14T04:50:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
65832024-11-14T04:50:45.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
65842024-11-14T04:50:45.327ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65852024-11-14T04:50:45.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65862024-11-14T04:50:45.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65872024-11-14T04:50:45.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65882024-11-14T04:50:45.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65892024-11-14T04:50:45.329ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65902024-11-14T04:50:45.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
65912024-11-14T04:50:45.331ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65922024-11-14T04:50:45.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65932024-11-14T04:50:45.332ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65942024-11-14T04:50:45.332ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1224, tv_nsec: 106140546 } })
65952024-11-14T04:50:45.333ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
65962024-11-14T04:50:45.333ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65972024-11-14T04:50:45.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65982024-11-14T04:50:45.334ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65992024-11-14T04:50:45.334ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 15.313673ms iteration = 10 status = {"errors":[],"records_set_to_done":[]}
66002024-11-14T04:50:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
66012024-11-14T04:50:45.335ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66022024-11-14T04:50:45.335ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1224, tv_nsec: 108894005 } })
66032024-11-14T04:50:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66042024-11-14T04:50:45.335ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
66052024-11-14T04:50:45.337ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
66062024-11-14T04:50:45.337ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
66072024-11-14T04:50:45.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
66082024-11-14T04:50:45.340ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66092024-11-14T04:50:45.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66102024-11-14T04:50:45.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66112024-11-14T04:50:45.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66122024-11-14T04:50:45.342ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66132024-11-14T04:50:45.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66142024-11-14T04:50:45.345ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1224, tv_nsec: 119071220 } })
66152024-11-14T04:50:45.346ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
66162024-11-14T04:50:45.348ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66172024-11-14T04:50:45.349ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66182024-11-14T04:50:45.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66192024-11-14T04:50:45.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66202024-11-14T04:50:45.352ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
66212024-11-14T04:50:45.352ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66222024-11-14T04:50:45.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66232024-11-14T04:50:45.354ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
66242024-11-14T04:50:45.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66252024-11-14T04:50:45.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66262024-11-14T04:50:45.357ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
66272024-11-14T04:50:45.357ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 38.095103ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
66282024-11-14T04:50:45.387ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
66292024-11-14T04:50:45.387ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.913176ms iteration = 28 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
66302024-11-14T04:50:45.393ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 74.038114ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
66312024-11-14T04:50:45.394ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 75.596139ms iteration = 12 status = {"physical_disks_added":0}
66322024-11-14T04:50:45.400ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66332024-11-14T04:50:45.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66342024-11-14T04:50:45.402ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
66352024-11-14T04:50:45.403ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 83.995502ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
66362024-11-14T04:50:45.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
66372024-11-14T04:50:45.403ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
66382024-11-14T04:50:45.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
66392024-11-14T04:50:45.405ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 86.220627ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
66402024-11-14T04:50:45.406ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 86.622969ms iteration = 10 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
66412024-11-14T04:50:45.406ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
66422024-11-14T04:50:45.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
66432024-11-14T04:50:45.408ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 89.379938ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
66442024-11-14T04:50:45.409ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66452024-11-14T04:50:45.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66462024-11-14T04:50:45.412ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66472024-11-14T04:50:45.412ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66482024-11-14T04:50:45.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66492024-11-14T04:50:45.414ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
66502024-11-14T04:50:45.414ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66512024-11-14T04:50:45.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66522024-11-14T04:50:45.414ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66532024-11-14T04:50:45.414ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
66542024-11-14T04:50:45.414ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
66552024-11-14T04:50:45.414ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
66562024-11-14T04:50:45.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
66572024-11-14T04:50:45.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
66582024-11-14T04:50:45.418ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66592024-11-14T04:50:45.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66602024-11-14T04:50:45.418ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
66612024-11-14T04:50:45.418ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
66622024-11-14T04:50:45.418ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
66632024-11-14T04:50:45.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
66642024-11-14T04:50:45.420ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 101.17081ms iteration = 10 status = 0
66652024-11-14T04:50:45.421ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
66662024-11-14T04:50:45.421ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
66672024-11-14T04:50:45.421ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 102.132248ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
66682024-11-14T04:50:45.452ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
66692024-11-14T04:50:45.452ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66702024-11-14T04:50:45.452ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "44182b61-ad9b-4c9b-b3a9-4313d70cd96e", "content-length": "26", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
66712024-11-14T04:50:45.452ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66722024-11-14T04:50:45.457ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
66732024-11-14T04:50:45.482ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
66742024-11-14T04:50:45.482ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66752024-11-14T04:50:45.482ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fe0bc1a-23b5-4313-8323-81bbe44968b2", "content-length": "26", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
66762024-11-14T04:50:45.482ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66772024-11-14T04:50:45.484ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
66782024-11-14T04:50:45.484ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66792024-11-14T04:50:45.484ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c060293-63c6-4c8c-8a5f-5b04aeeff95e", "content-length": "26", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
66802024-11-14T04:50:45.484ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66812024-11-14T04:50:45.484ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
66822024-11-14T04:50:45.486ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
66832024-11-14T04:50:45.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66842024-11-14T04:50:45.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66852024-11-14T04:50:45.532ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
66862024-11-14T04:50:45.532ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66872024-11-14T04:50:45.532ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b927c6e-a931-4019-97fb-f907e2ebee76", "content-length": "372", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
66882024-11-14T04:50:45.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66892024-11-14T04:50:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66902024-11-14T04:50:45.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66912024-11-14T04:50:45.535ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66922024-11-14T04:50:45.535ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66932024-11-14T04:50:45.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66942024-11-14T04:50:45.537ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66952024-11-14T04:50:45.540ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66962024-11-14T04:50:45.540ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66972024-11-14T04:50:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66982024-11-14T04:50:45.546ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
66992024-11-14T04:50:45.546ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
67002024-11-14T04:50:45.547ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "978824f9-cacd-4177-a17b-7d0cccf554d1", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
67012024-11-14T04:50:45.547ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67022024-11-14T04:50:45.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67032024-11-14T04:50:45.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67042024-11-14T04:50:45.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67052024-11-14T04:50:45.552ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67062024-11-14T04:50:45.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67072024-11-14T04:50:45.557ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
67082024-11-14T04:50:45.557ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
67092024-11-14T04:50:45.557ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f84ad85d-469d-4883-8c83-1ddb490d536c", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
67102024-11-14T04:50:45.559ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 239.624119ms iteration = 11 status = {}
67112024-11-14T04:50:45.564ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 245.227921ms iteration = 10 status = {}
67122024-11-14T04:50:45.568ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
67132024-11-14T04:50:45.568ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e75875fd-b805-419e-9859-80080bfbc7f4", "content-length": "1", "date": "Thu, 14 Nov 2024 04:50:44 GMT"} })
67142024-11-14T04:50:45.569ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
67152024-11-14T04:50:45.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
67162024-11-14T04:50:45.571ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 252.515692ms iteration = 10 status = null
67172024-11-14T04:50:45.659ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67182024-11-14T04:50:45.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67192024-11-14T04:50:45.666ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
67202024-11-14T04:50:45.667ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
67212024-11-14T04:50:45.667ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67222024-11-14T04:50:45.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67232024-11-14T04:50:45.676ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
67242024-11-14T04:50:45.676ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e2673f3-ee6f-4659-a095-20bd762fd057", "content-length": "134", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
67252024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
67262024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
67272024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
67282024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
67292024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
67302024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
67312024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
67322024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
67332024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
67342024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
67352024-11-14T04:50:45.676ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
67362024-11-14T04:50:45.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67372024-11-14T04:50:45.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67382024-11-14T04:50:45.678ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
67392024-11-14T04:50:45.678ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "14819143-1614-4479-9c2a-4b84bb4eeb86", "content-length": "2", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
67402024-11-14T04:50:45.678ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
67412024-11-14T04:50:45.678ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "23dd3a57-7519-4a74-ad5f-062185aace94", "content-length": "51", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
67422024-11-14T04:50:45.678ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
67432024-11-14T04:50:45.678ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
67442024-11-14T04:50:45.739ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "22c4d1ae-70f3-4ad2-84ea-bc9111958acc", "content-length": "691", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
67452024-11-14T04:50:45.739ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
67462024-11-14T04:50:45.739ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
67472024-11-14T04:50:45.739ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
67482024-11-14T04:50:45.741ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 422.374421ms iteration = 10 status = {}
67492024-11-14T04:50:45.913ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d627193f-be5e-4f0a-a3bb-7e2097b9f773", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
67502024-11-14T04:50:45.913ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
67512024-11-14T04:50:45.913ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
67522024-11-14T04:50:45.913ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5543d67b-b447-4e15-bc05-b0cfd03d7e05", "content-length": "682", "date": "Thu, 14 Nov 2024 04:50:45 GMT"} })
67532024-11-14T04:50:45.913ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67542024-11-14T04:50:45.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67552024-11-14T04:50:45.917ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67562024-11-14T04:50:45.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67572024-11-14T04:50:45.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67582024-11-14T04:50:45.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67592024-11-14T04:50:45.922ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67602024-11-14T04:50:45.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67612024-11-14T04:50:45.929ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 610.134798ms iteration = 10 status = {}
67622024-11-14T04:50:50.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1171 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = f2458b8e-1414-4042-bacf-aeceac4b0c68 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
67632024-11-14T04:50:55.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
67642024-11-14T04:50:55.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67652024-11-14T04:50:55.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
67662024-11-14T04:50:55.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67672024-11-14T04:50:55.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67682024-11-14T04:50:55.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67692024-11-14T04:50:55.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67702024-11-14T04:50:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67712024-11-14T04:50:55.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67722024-11-14T04:50:55.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.886742ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67732024-11-14T04:50:55.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67742024-11-14T04:50:55.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67752024-11-14T04:50:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67762024-11-14T04:50:55.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67772024-11-14T04:50:55.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67782024-11-14T04:50:55.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67792024-11-14T04:50:55.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
67802024-11-14T04:50:55.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67812024-11-14T04:50:55.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67822024-11-14T04:50:55.338ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
67832024-11-14T04:50:55.338ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.986189ms iteration = 29 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
67842024-11-14T04:51:05.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1285 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 4ee8a16b-15da-40bd-9b21-5ed01623904c response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
67852024-11-14T04:51:05.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
67862024-11-14T04:51:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67872024-11-14T04:51:05.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
67882024-11-14T04:51:05.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67892024-11-14T04:51:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67902024-11-14T04:51:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67912024-11-14T04:51:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67922024-11-14T04:51:05.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67932024-11-14T04:51:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67942024-11-14T04:51:05.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.540521ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67952024-11-14T04:51:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67962024-11-14T04:51:05.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67972024-11-14T04:51:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67982024-11-14T04:51:05.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67992024-11-14T04:51:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68002024-11-14T04:51:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68012024-11-14T04:51:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
68022024-11-14T04:51:05.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68032024-11-14T04:51:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68042024-11-14T04:51:05.336ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
68052024-11-14T04:51:05.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.387243ms iteration = 30 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
68062024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
68072024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
68082024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
68092024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
68102024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
68112024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
68122024-11-14T04:51:15.319ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:41:15.319108951 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
68132024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
68142024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
68152024-11-14T04:51:15.319ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
68162024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
68172024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.15\u{b5}s iteration = 7 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
68182024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
68192024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
68202024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
68212024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
68222024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
68232024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
68242024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
68252024-11-14T04:51:15.319ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
68262024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
68272024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
68282024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
68292024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
68302024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68312024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68322024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
68332024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
68342024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
68352024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68362024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
68372024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
68382024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
68392024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
68402024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
68412024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
68422024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
68432024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
68442024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68452024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
68462024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
68472024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
68482024-11-14T04:51:15.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
68492024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68502024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
68512024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
68522024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68532024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68542024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
68552024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68562024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
68572024-11-14T04:51:15.319ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
68582024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
68592024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
68602024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
68612024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
68622024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
68632024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
68642024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
68652024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
68662024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
68672024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
68682024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
68692024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
68702024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
68712024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
68722024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
68732024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
68742024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
68752024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
68762024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
68772024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
68782024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68792024-11-14T04:51:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
68802024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
68812024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
68822024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68832024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68842024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68852024-11-14T04:51:15.320ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1254, tv_nsec: 93682051 } })
68862024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
68872024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68882024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68892024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
68902024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
68912024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68922024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
68932024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
68942024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
68952024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
68962024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
68972024-11-14T04:51:15.320ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1254, tv_nsec: 93817355 } })
68982024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
68992024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.213695ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
69002024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
69012024-11-14T04:51:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.207445ms iteration = 7 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
69022024-11-14T04:51:15.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69032024-11-14T04:51:15.322ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:41:15.319108951 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
69042024-11-14T04:51:15.322ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.464501ms iteration = 6 status = {"expiration":"2024-11-14T04:41:15.319108951Z","pruned":[]}
69052024-11-14T04:51:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
69062024-11-14T04:51:15.325ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
69072024-11-14T04:51:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
69082024-11-14T04:51:15.328ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1254, tv_nsec: 101784925 } })
69092024-11-14T04:51:15.328ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
69102024-11-14T04:51:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
69112024-11-14T04:51:15.328ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
69122024-11-14T04:51:15.328ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69132024-11-14T04:51:15.328ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.601098ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
69142024-11-14T04:51:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69152024-11-14T04:51:15.329ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.00172ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
69162024-11-14T04:51:15.330ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69172024-11-14T04:51:15.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69182024-11-14T04:51:15.332ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69192024-11-14T04:51:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
69202024-11-14T04:51:15.334ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69212024-11-14T04:51:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69222024-11-14T04:51:15.335ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69232024-11-14T04:51:15.335ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
69242024-11-14T04:51:15.335ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
69252024-11-14T04:51:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
69262024-11-14T04:51:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69272024-11-14T04:51:15.335ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69282024-11-14T04:51:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69292024-11-14T04:51:15.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
69302024-11-14T04:51:15.336ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69312024-11-14T04:51:15.336ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69322024-11-14T04:51:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69332024-11-14T04:51:15.338ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1254, tv_nsec: 112722102 } })
69342024-11-14T04:51:15.339ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
69352024-11-14T04:51:15.339ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69362024-11-14T04:51:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69372024-11-14T04:51:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
69382024-11-14T04:51:15.339ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69392024-11-14T04:51:15.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69402024-11-14T04:51:15.342ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69412024-11-14T04:51:15.342ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 22.79096ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
69422024-11-14T04:51:15.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69432024-11-14T04:51:15.346ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69442024-11-14T04:51:15.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69452024-11-14T04:51:15.348ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69462024-11-14T04:51:15.349ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
69472024-11-14T04:51:15.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
69482024-11-14T04:51:15.350ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69492024-11-14T04:51:15.351ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
69502024-11-14T04:51:15.351ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
69512024-11-14T04:51:15.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
69522024-11-14T04:51:15.353ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
69532024-11-14T04:51:15.353ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69542024-11-14T04:51:15.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
69552024-11-14T04:51:15.355ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
69562024-11-14T04:51:15.355ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 36.213769ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
69572024-11-14T04:51:15.361ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69582024-11-14T04:51:15.361ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69592024-11-14T04:51:15.361ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
69602024-11-14T04:51:15.361ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
69612024-11-14T04:51:15.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
69622024-11-14T04:51:15.366ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
69632024-11-14T04:51:15.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69642024-11-14T04:51:15.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69652024-11-14T04:51:15.366ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1254, tv_nsec: 140335562 } })
69662024-11-14T04:51:15.366ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
69672024-11-14T04:51:15.371ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69682024-11-14T04:51:15.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69692024-11-14T04:51:15.375ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69702024-11-14T04:51:15.375ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
69712024-11-14T04:51:15.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
69722024-11-14T04:51:15.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
69732024-11-14T04:51:15.381ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69742024-11-14T04:51:15.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69752024-11-14T04:51:15.383ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69762024-11-14T04:51:15.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69772024-11-14T04:51:15.384ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69782024-11-14T04:51:15.384ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69792024-11-14T04:51:15.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
69802024-11-14T04:51:15.384ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
69812024-11-14T04:51:15.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
69822024-11-14T04:51:15.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69832024-11-14T04:51:15.388ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69842024-11-14T04:51:15.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69852024-11-14T04:51:15.391ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69862024-11-14T04:51:15.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
69872024-11-14T04:51:15.394ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69882024-11-14T04:51:15.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69892024-11-14T04:51:15.398ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69902024-11-14T04:51:15.400ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 80.958814ms iteration = 11 status = {"errors":[],"records_set_to_done":[]}
69912024-11-14T04:51:15.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
69922024-11-14T04:51:15.403ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69932024-11-14T04:51:15.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
69942024-11-14T04:51:15.406ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69952024-11-14T04:51:15.410ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
69962024-11-14T04:51:15.410ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69972024-11-14T04:51:15.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69982024-11-14T04:51:15.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69992024-11-14T04:51:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
70002024-11-14T04:51:15.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70012024-11-14T04:51:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70022024-11-14T04:51:15.424ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
70032024-11-14T04:51:15.424ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
70042024-11-14T04:51:15.424ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 105.349281ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
70052024-11-14T04:51:15.427ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
70062024-11-14T04:51:15.427ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
70072024-11-14T04:51:15.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
70082024-11-14T04:51:15.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
70092024-11-14T04:51:15.432ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
70102024-11-14T04:51:15.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
70112024-11-14T04:51:15.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
70122024-11-14T04:51:15.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
70132024-11-14T04:51:15.447ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 128.269115ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
70142024-11-14T04:51:15.452ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
70152024-11-14T04:51:15.452ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
70162024-11-14T04:51:15.452ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 132.61485ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
70172024-11-14T04:51:15.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70182024-11-14T04:51:15.456ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
70192024-11-14T04:51:15.457ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
70202024-11-14T04:51:15.457ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 137.664847ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
70212024-11-14T04:51:15.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
70222024-11-14T04:51:15.458ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
70232024-11-14T04:51:15.458ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
70242024-11-14T04:51:15.458ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
70252024-11-14T04:51:15.458ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70262024-11-14T04:51:15.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70272024-11-14T04:51:15.467ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70282024-11-14T04:51:15.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70292024-11-14T04:51:15.473ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70302024-11-14T04:51:15.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70312024-11-14T04:51:15.499ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
70322024-11-14T04:51:15.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
70332024-11-14T04:51:15.504ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 184.851723ms iteration = 13 status = {"physical_disks_added":0}
70342024-11-14T04:51:15.507ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
70352024-11-14T04:51:15.514ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
70362024-11-14T04:51:15.514ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
70372024-11-14T04:51:15.514ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0243e126-df55-4087-a570-15c2e8c2d6bb", "content-length": "26", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
70382024-11-14T04:51:15.514ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
70392024-11-14T04:51:15.517ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
70402024-11-14T04:51:15.521ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
70412024-11-14T04:51:15.521ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
70422024-11-14T04:51:15.521ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
70432024-11-14T04:51:15.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70442024-11-14T04:51:15.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
70452024-11-14T04:51:15.708ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
70462024-11-14T04:51:15.717ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
70472024-11-14T04:51:15.718ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e2a6da7-c95c-4f84-b595-e5ee90e8a392", "content-length": "691", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
70482024-11-14T04:51:15.718ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
70492024-11-14T04:51:15.718ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
70502024-11-14T04:51:15.718ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
70512024-11-14T04:51:15.720ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 401.22998ms iteration = 11 status = {}
70522024-11-14T04:51:15.721ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
70532024-11-14T04:51:15.721ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
70542024-11-14T04:51:15.721ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "476d611d-cf2e-4c7e-a24a-2652642d0f17", "content-length": "26", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
70552024-11-14T04:51:15.721ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
70562024-11-14T04:51:15.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70572024-11-14T04:51:15.723ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
70582024-11-14T04:51:15.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70592024-11-14T04:51:15.729ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
70602024-11-14T04:51:15.734ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
70612024-11-14T04:51:15.734ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
70622024-11-14T04:51:15.734ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1c10bce-4a77-49aa-9c94-f5bfb4353ffc", "content-length": "26", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
70632024-11-14T04:51:15.734ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
70642024-11-14T04:51:15.736ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
70652024-11-14T04:51:15.856ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 536.765983ms iteration = 11 status = {}
70662024-11-14T04:51:15.899ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
70672024-11-14T04:51:15.900ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
70682024-11-14T04:51:15.900ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
70692024-11-14T04:51:15.900ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "3069188a-03d6-4190-a104-c8e6bcab7656", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
70702024-11-14T04:51:15.900ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
70712024-11-14T04:51:15.900ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecd5c202-38c5-4b49-9879-520f00303080", "content-length": "372", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
70722024-11-14T04:51:15.900ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70732024-11-14T04:51:15.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70742024-11-14T04:51:15.902ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
70752024-11-14T04:51:15.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
70762024-11-14T04:51:15.903ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "7d2d1c9f-7be1-44de-846c-eaeb8d3ee5f6", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
70772024-11-14T04:51:15.903ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
70782024-11-14T04:51:15.904ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70792024-11-14T04:51:15.905ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70802024-11-14T04:51:15.905ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3dbb3f49-d4f1-480c-bdae-339a17f288d5", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
70812024-11-14T04:51:15.905ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
70822024-11-14T04:51:15.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70832024-11-14T04:51:15.905ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70842024-11-14T04:51:15.907ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
70852024-11-14T04:51:15.907ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 588.557104ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
70862024-11-14T04:51:15.908ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
70872024-11-14T04:51:15.908ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70882024-11-14T04:51:15.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70892024-11-14T04:51:15.910ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
70902024-11-14T04:51:15.910ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70912024-11-14T04:51:15.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
70922024-11-14T04:51:15.932ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
70932024-11-14T04:51:15.932ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "33079b6a-cc54-454b-8a16-6b535c34c9f9", "content-length": "1", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
70942024-11-14T04:51:15.935ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
70952024-11-14T04:51:15.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
70962024-11-14T04:51:15.937ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
70972024-11-14T04:51:15.937ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b7679350-485f-4514-a9c6-a4f34e6688a3", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
70982024-11-14T04:51:15.937ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
70992024-11-14T04:51:15.939ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 619.961873ms iteration = 11 status = null
71002024-11-14T04:51:15.939ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ca494f65-f95a-4384-8c53-f9ffcf4c249b", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
71012024-11-14T04:51:15.939ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
71022024-11-14T04:51:15.941ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
71032024-11-14T04:51:15.941ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 621.766785ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
71042024-11-14T04:51:15.951ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 632.459095ms iteration = 11 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
71052024-11-14T04:51:15.952ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71062024-11-14T04:51:15.953ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
71072024-11-14T04:51:15.953ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
71082024-11-14T04:51:15.953ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71092024-11-14T04:51:15.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71102024-11-14T04:51:15.954ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71112024-11-14T04:51:15.955ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 636.515522ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
71122024-11-14T04:51:15.956ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
71132024-11-14T04:51:15.956ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71142024-11-14T04:51:15.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71152024-11-14T04:51:15.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
71162024-11-14T04:51:15.957ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71172024-11-14T04:51:15.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71182024-11-14T04:51:15.957ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71192024-11-14T04:51:15.958ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71202024-11-14T04:51:15.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71212024-11-14T04:51:15.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71222024-11-14T04:51:15.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
71232024-11-14T04:51:15.960ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71242024-11-14T04:51:15.960ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
71252024-11-14T04:51:15.960ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
71262024-11-14T04:51:15.960ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71272024-11-14T04:51:15.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71282024-11-14T04:51:15.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71292024-11-14T04:51:15.960ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 641.327771ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
71302024-11-14T04:51:15.961ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71312024-11-14T04:51:15.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71322024-11-14T04:51:15.962ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71332024-11-14T04:51:15.962ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 643.249047ms iteration = 11 status = 0
71342024-11-14T04:51:15.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
71352024-11-14T04:51:15.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71362024-11-14T04:51:15.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71372024-11-14T04:51:15.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
71382024-11-14T04:51:15.965ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71392024-11-14T04:51:15.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71402024-11-14T04:51:15.967ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
71412024-11-14T04:51:15.967ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
71422024-11-14T04:51:15.967ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
71432024-11-14T04:51:15.967ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 648.336154ms iteration = 7 status = {"generation":2}
71442024-11-14T04:51:15.977ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
71452024-11-14T04:51:15.977ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 658.121647ms iteration = 31 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
71462024-11-14T04:51:15.980ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
71472024-11-14T04:51:15.981ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
71482024-11-14T04:51:15.985ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71492024-11-14T04:51:15.988ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
71502024-11-14T04:51:15.988ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71512024-11-14T04:51:15.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71522024-11-14T04:51:15.989ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
71532024-11-14T04:51:15.989ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 669.585579ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
71542024-11-14T04:51:15.991ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
71552024-11-14T04:51:15.991ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
71562024-11-14T04:51:15.991ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
71572024-11-14T04:51:15.991ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 671.700021ms iteration = 7 status = {"generation":1}
71582024-11-14T04:51:15.991ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71592024-11-14T04:51:15.991ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71602024-11-14T04:51:15.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71612024-11-14T04:51:15.993ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
71622024-11-14T04:51:15.993ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71632024-11-14T04:51:15.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71642024-11-14T04:51:15.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
71652024-11-14T04:51:15.994ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71662024-11-14T04:51:15.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71672024-11-14T04:51:15.997ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71682024-11-14T04:51:15.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71692024-11-14T04:51:15.999ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71702024-11-14T04:51:15.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71712024-11-14T04:51:16.000ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71722024-11-14T04:51:16.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71732024-11-14T04:51:16.002ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71742024-11-14T04:51:16.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71752024-11-14T04:51:16.003ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
71762024-11-14T04:51:16.003ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "43cc09f0-a784-42c3-8d2d-775693d3f821", "content-length": "134", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
71772024-11-14T04:51:16.003ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
71782024-11-14T04:51:16.003ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
71792024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
71802024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
71812024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
71822024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
71832024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
71842024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
71852024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
71862024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
71872024-11-14T04:51:16.004ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
71882024-11-14T04:51:16.004ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71892024-11-14T04:51:16.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71902024-11-14T04:51:16.013ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
71912024-11-14T04:51:16.014ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd6b9289-8f27-4169-8e16-092c9f90f1bf", "content-length": "2", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
71922024-11-14T04:51:16.014ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
71932024-11-14T04:51:16.014ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c9b3b75-bde2-4274-9bed-028f09a22101", "content-length": "51", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
71942024-11-14T04:51:16.014ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
71952024-11-14T04:51:16.014ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
71962024-11-14T04:51:16.023ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
71972024-11-14T04:51:16.023ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 704.527392ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
71982024-11-14T04:51:16.041ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71992024-11-14T04:51:16.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72002024-11-14T04:51:16.062ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ffd5304d-dfe5-4b95-85ec-d8f24d8f7c93", "date": "Thu, 14 Nov 2024 04:51:16 GMT"} })
72012024-11-14T04:51:16.062ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72022024-11-14T04:51:16.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72032024-11-14T04:51:16.065ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72042024-11-14T04:51:16.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72052024-11-14T04:51:16.068ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72062024-11-14T04:51:16.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72072024-11-14T04:51:16.071ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72082024-11-14T04:51:16.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72092024-11-14T04:51:16.072ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
72102024-11-14T04:51:16.072ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
72112024-11-14T04:51:16.073ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72122024-11-14T04:51:16.075ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "929ce573-9b88-4b5d-999d-8e3b1fff3112", "date": "Thu, 14 Nov 2024 04:51:16 GMT"} })
72132024-11-14T04:51:16.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
72142024-11-14T04:51:16.076ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72152024-11-14T04:51:16.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72162024-11-14T04:51:16.077ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 758.628338ms iteration = 12 status = {}
72172024-11-14T04:51:16.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72182024-11-14T04:51:16.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72192024-11-14T04:51:16.080ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72202024-11-14T04:51:16.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72212024-11-14T04:51:16.086ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72222024-11-14T04:51:16.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72232024-11-14T04:51:16.090ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72242024-11-14T04:51:16.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72252024-11-14T04:51:16.091ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72262024-11-14T04:51:16.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72272024-11-14T04:51:16.093ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72282024-11-14T04:51:16.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72292024-11-14T04:51:16.094ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72302024-11-14T04:51:16.100ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
72312024-11-14T04:51:16.100ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72322024-11-14T04:51:16.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72332024-11-14T04:51:16.101ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72342024-11-14T04:51:16.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72352024-11-14T04:51:16.103ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72362024-11-14T04:51:16.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72372024-11-14T04:51:16.104ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72382024-11-14T04:51:16.108ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
72392024-11-14T04:51:16.108ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72402024-11-14T04:51:16.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72412024-11-14T04:51:16.114ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72422024-11-14T04:51:16.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72432024-11-14T04:51:16.115ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72442024-11-14T04:51:16.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72452024-11-14T04:51:16.117ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72462024-11-14T04:51:16.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72472024-11-14T04:51:16.119ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72482024-11-14T04:51:16.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72492024-11-14T04:51:16.122ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72502024-11-14T04:51:16.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72512024-11-14T04:51:16.123ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72522024-11-14T04:51:16.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72532024-11-14T04:51:16.125ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72542024-11-14T04:51:16.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72552024-11-14T04:51:16.126ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72562024-11-14T04:51:16.131ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
72572024-11-14T04:51:16.131ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72582024-11-14T04:51:16.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72592024-11-14T04:51:16.135ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72602024-11-14T04:51:16.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72612024-11-14T04:51:16.138ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72622024-11-14T04:51:16.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72632024-11-14T04:51:16.140ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72642024-11-14T04:51:16.144ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
72652024-11-14T04:51:16.144ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72662024-11-14T04:51:16.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72672024-11-14T04:51:16.151ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 68cb12a7-f65f-4940-ad7a-12ac0e8fa220, kind: Service { id: 7d1b51fe-adf1-4350-9c25-9240054e3911 }, name: Name("nexus-7d1b51fe-adf1-4350-9c25-9240054e3911"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 137, 76])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5946ca8-a389-4a86-aed2-97d4e8639c67, kind: Service { id: 13db289d-f9b6-41e9-b3bd-5ab4f3105694 }, name: Name("nexus-13db289d-f9b6-41e9-b3bd-5ab4f3105694"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b897d57d-b436-467c-89e9-441e43e441fa, kind: Service { id: 79994e02-6747-4071-83af-7872b31640fb }, name: Name("nexus-79994e02-6747-4071-83af-7872b31640fb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 66])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 32de9250-63c9-46be-9a6b-ee4af19ab1ab, kind: Service { id: 48221b8a-d832-4d05-ae47-e92f81388c0c }, name: Name("external-dns-48221b8a-d832-4d05-ae47-e92f81388c0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 126])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: da2f10b4-11b4-4931-a7f7-d415353256ec, kind: Service { id: 8c0d54f6-76b3-47cc-9828-b8e677cc2961 }, name: Name("external-dns-8c0d54f6-76b3-47cc-9828-b8e677cc2961"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 129])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
72682024-11-14T04:51:16.151ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72692024-11-14T04:51:16.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72702024-11-14T04:51:16.153ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72712024-11-14T04:51:16.155ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
72722024-11-14T04:51:16.155ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72732024-11-14T04:51:16.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72742024-11-14T04:51:16.157ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
72752024-11-14T04:51:16.157ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 32de9250-63c9-46be-9a6b-ee4af19ab1ab, kind: Service { id: 48221b8a-d832-4d05-ae47-e92f81388c0c }, name: Name("external-dns-48221b8a-d832-4d05-ae47-e92f81388c0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 126])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: da2f10b4-11b4-4931-a7f7-d415353256ec, kind: Service { id: 8c0d54f6-76b3-47cc-9828-b8e677cc2961 }, name: Name("external-dns-8c0d54f6-76b3-47cc-9828-b8e677cc2961"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 129])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 68cb12a7-f65f-4940-ad7a-12ac0e8fa220, kind: Service { id: 7d1b51fe-adf1-4350-9c25-9240054e3911 }, name: Name("nexus-7d1b51fe-adf1-4350-9c25-9240054e3911"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 137, 76])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5946ca8-a389-4a86-aed2-97d4e8639c67, kind: Service { id: 13db289d-f9b6-41e9-b3bd-5ab4f3105694 }, name: Name("nexus-13db289d-f9b6-41e9-b3bd-5ab4f3105694"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b897d57d-b436-467c-89e9-441e43e441fa, kind: Service { id: 79994e02-6747-4071-83af-7872b31640fb }, name: Name("nexus-79994e02-6747-4071-83af-7872b31640fb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 66])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
72762024-11-14T04:51:16.157ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
72772024-11-14T04:51:16.274ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: c073d51c-abaa-48df-abb6-2f053e9c1bec, time_created: 2024-11-14T04:46:13.572725Z, time_modified: 2024-11-14T04:46:13.572725Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: f971eb12-0fea-4084-96d3-74198bddd289, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
72782024-11-14T04:51:16.274ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
72792024-11-14T04:51:16.274ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72802024-11-14T04:51:16.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72812024-11-14T04:51:16.277ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72822024-11-14T04:51:16.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72832024-11-14T04:51:16.279ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72842024-11-14T04:51:16.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
72852024-11-14T04:51:16.289ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "34995683-043a-4c88-92af-5e02b8f13773", "date": "Thu, 14 Nov 2024 04:51:16 GMT"} })
72862024-11-14T04:51:16.289ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
72872024-11-14T04:51:16.289ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
72882024-11-14T04:51:16.290ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d4bf619-cd93-4697-9ebb-2de825e6592e", "content-length": "682", "date": "Thu, 14 Nov 2024 04:51:15 GMT"} })
72892024-11-14T04:51:16.290ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72902024-11-14T04:51:16.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72912024-11-14T04:51:16.294ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72922024-11-14T04:51:16.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72932024-11-14T04:51:16.297ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72942024-11-14T04:51:16.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72952024-11-14T04:51:16.300ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72962024-11-14T04:51:16.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72972024-11-14T04:51:16.307ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 988.054802ms iteration = 11 status = {}
72982024-11-14T04:51:16.313ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
72992024-11-14T04:51:16.313ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "fc89927a-8bf9-4b0e-a3a3-770b256c86d7", "date": "Thu, 14 Nov 2024 04:51:16 GMT"} })
73002024-11-14T04:51:16.315ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
73012024-11-14T04:51:16.315ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 996.606449ms
73022024-11-14T04:51:16.315ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 996.613979ms iteration = 2 status = {}
73032024-11-14T04:51:20.081ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
73042024-11-14T04:51:20.081ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
73052024-11-14T04:51:20.112ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
73062024-11-14T04:51:20.112ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:50335
73072024-11-14T04:51:20.112ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:50335 req_id = 650ab066-4f74-4df3-b780-e2dfd7a95034 roles = RoleSet { roles: {} } uri = /metrics/producers
73082024-11-14T04:51:20.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:50335 req_id = 650ab066-4f74-4df3-b780-e2dfd7a95034 resource = Database result = Ok(()) uri = /metrics/producers
73092024-11-14T04:51:20.152ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
73102024-11-14T04:51:20.152ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40172 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:50335 req_id = 650ab066-4f74-4df3-b780-e2dfd7a95034 response_code = 201 uri = /metrics/producers
73112024-11-14T04:51:20.152ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "650ab066-4f74-4df3-b780-e2dfd7a95034", "content-length": "41", "date": "Thu, 14 Nov 2024 04:51:20 GMT"} })
73122024-11-14T04:51:20.155ZDEBGnexus (producer-registration-task): registered with nexus successfully
73132024-11-14T04:51:20.155ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
73142024-11-14T04:51:20.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1093 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = b4bf3aab-f553-455f-b860-1a94e5c500be response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
73152024-11-14T04:51:25.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
73162024-11-14T04:51:25.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73172024-11-14T04:51:25.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
73182024-11-14T04:51:25.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73192024-11-14T04:51:25.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73202024-11-14T04:51:25.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73212024-11-14T04:51:25.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73222024-11-14T04:51:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
73232024-11-14T04:51:25.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73242024-11-14T04:51:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73252024-11-14T04:51:25.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73262024-11-14T04:51:25.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73272024-11-14T04:51:25.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73282024-11-14T04:51:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
73292024-11-14T04:51:25.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73302024-11-14T04:51:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73312024-11-14T04:51:25.340ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
73322024-11-14T04:51:25.340ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.523303ms iteration = 32 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
73332024-11-14T04:51:25.348ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73342024-11-14T04:51:25.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73352024-11-14T04:51:25.378ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.40334ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
73362024-11-14T04:51:35.261ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1048 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 8b90f8f1-7f52-4225-8e4e-d62591e48efb response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
73372024-11-14T04:51:35.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
73382024-11-14T04:51:35.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73392024-11-14T04:51:35.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
73402024-11-14T04:51:35.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73412024-11-14T04:51:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73422024-11-14T04:51:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73432024-11-14T04:51:35.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73442024-11-14T04:51:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
73452024-11-14T04:51:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73462024-11-14T04:51:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73472024-11-14T04:51:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73482024-11-14T04:51:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73492024-11-14T04:51:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73502024-11-14T04:51:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
73512024-11-14T04:51:35.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73522024-11-14T04:51:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73532024-11-14T04:51:35.339ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
73542024-11-14T04:51:35.339ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.638287ms iteration = 33 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
73552024-11-14T04:51:35.352ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73562024-11-14T04:51:35.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73572024-11-14T04:51:35.354ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.832027ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
73582024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
73592024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73602024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
73612024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73622024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
73632024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
73642024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
73652024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
73662024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
73672024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
73682024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
73692024-11-14T04:51:45.318ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
73702024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
73712024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73722024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
73732024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
73742024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
73752024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
73762024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
73772024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
73782024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73792024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
73802024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73812024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
73822024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73832024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
73842024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73852024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73862024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73872024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
73882024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
73892024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73902024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73912024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
73922024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73932024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
73942024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
73952024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73962024-11-14T04:51:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73972024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
73982024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
73992024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
74002024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
74012024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
74022024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
74032024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
74042024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
74052024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
74062024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
74072024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
74082024-11-14T04:51:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
74092024-11-14T04:51:45.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74102024-11-14T04:51:45.325ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1284, tv_nsec: 99443555 } })
74112024-11-14T04:51:45.325ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1284, tv_nsec: 99529858 } })
74122024-11-14T04:51:45.325ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
74132024-11-14T04:51:45.326ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
74142024-11-14T04:51:45.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74152024-11-14T04:51:45.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74162024-11-14T04:51:45.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74172024-11-14T04:51:45.329ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74182024-11-14T04:51:45.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
74192024-11-14T04:51:45.331ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74202024-11-14T04:51:45.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74212024-11-14T04:51:45.331ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 13.825131ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
74222024-11-14T04:51:45.332ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74232024-11-14T04:51:45.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74242024-11-14T04:51:45.334ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74252024-11-14T04:51:45.335ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74262024-11-14T04:51:45.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
74272024-11-14T04:51:45.338ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
74282024-11-14T04:51:45.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
74292024-11-14T04:51:45.338ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74302024-11-14T04:51:45.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
74312024-11-14T04:51:45.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74322024-11-14T04:51:45.344ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74332024-11-14T04:51:45.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74342024-11-14T04:51:45.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74352024-11-14T04:51:45.346ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
74362024-11-14T04:51:45.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
74372024-11-14T04:51:45.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
74382024-11-14T04:51:45.347ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74392024-11-14T04:51:45.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74402024-11-14T04:51:45.348ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1284, tv_nsec: 122216555 } })
74412024-11-14T04:51:45.348ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
74422024-11-14T04:51:45.394ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
74432024-11-14T04:51:45.394ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
74442024-11-14T04:51:45.394ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8569137a-3b19-4529-b714-cdea452d17a9", "content-length": "26", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
74452024-11-14T04:51:45.394ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
74462024-11-14T04:51:45.397ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
74472024-11-14T04:51:45.400ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74482024-11-14T04:51:45.400ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
74492024-11-14T04:51:45.402ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
74502024-11-14T04:51:45.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
74512024-11-14T04:51:45.403ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
74522024-11-14T04:51:45.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
74532024-11-14T04:51:45.404ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7284e10f-5fad-466e-ba91-d5cef1badac4", "content-length": "26", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
74542024-11-14T04:51:45.404ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
74552024-11-14T04:51:45.406ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
74562024-11-14T04:51:45.406ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
74572024-11-14T04:51:45.406ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
74582024-11-14T04:51:45.406ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
74592024-11-14T04:51:45.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
74602024-11-14T04:51:45.408ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 89.540693ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
74612024-11-14T04:51:45.422ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
74622024-11-14T04:51:45.422ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74632024-11-14T04:51:45.422ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
74642024-11-14T04:51:45.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
74652024-11-14T04:51:45.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
74662024-11-14T04:51:45.423ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
74672024-11-14T04:51:45.423ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 105.855495ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
74682024-11-14T04:51:45.425ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 107.466612ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74692024-11-14T04:51:45.425ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
74702024-11-14T04:51:45.425ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 106.565426ms iteration = 12 status = {"errors":[],"records_set_to_done":[]}
74712024-11-14T04:51:45.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
74722024-11-14T04:51:45.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74732024-11-14T04:51:45.426ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
74742024-11-14T04:51:45.426ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
74752024-11-14T04:51:45.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
74762024-11-14T04:51:45.427ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74772024-11-14T04:51:45.429ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
74782024-11-14T04:51:45.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
74792024-11-14T04:51:45.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
74802024-11-14T04:51:45.430ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
74812024-11-14T04:51:45.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
74822024-11-14T04:51:45.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74832024-11-14T04:51:45.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
74842024-11-14T04:51:45.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74852024-11-14T04:51:45.433ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
74862024-11-14T04:51:45.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
74872024-11-14T04:51:45.452ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
74882024-11-14T04:51:45.452ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 134.850785ms iteration = 34 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
74892024-11-14T04:51:45.453ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
74902024-11-14T04:51:45.453ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
74912024-11-14T04:51:45.453ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "71a59d3f-e9f2-4661-8a7a-a741fc736ff4", "content-length": "26", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
74922024-11-14T04:51:45.453ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
74932024-11-14T04:51:45.455ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
74942024-11-14T04:51:45.471ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
74952024-11-14T04:51:45.471ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
74962024-11-14T04:51:45.471ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
74972024-11-14T04:51:45.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
74982024-11-14T04:51:45.472ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 154.238266ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
74992024-11-14T04:51:45.475ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75002024-11-14T04:51:45.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75012024-11-14T04:51:45.479ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 160.248311ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
75022024-11-14T04:51:45.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75032024-11-14T04:51:45.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75042024-11-14T04:51:45.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75052024-11-14T04:51:45.484ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 165.474202ms iteration = 12 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
75062024-11-14T04:51:45.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
75072024-11-14T04:51:45.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75082024-11-14T04:51:45.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75092024-11-14T04:51:45.488ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 170.328593ms iteration = 12 status = 0
75102024-11-14T04:51:45.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75112024-11-14T04:51:45.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75122024-11-14T04:51:45.508ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
75132024-11-14T04:51:45.508ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "05440b55-ba34-4cd2-8ba1-b8198e3832b3", "content-length": "1", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75142024-11-14T04:51:45.510ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
75152024-11-14T04:51:45.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
75162024-11-14T04:51:45.516ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 198.082366ms iteration = 12 status = null
75172024-11-14T04:51:45.520ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 202.149074ms iteration = 12 status = {}
75182024-11-14T04:51:45.523ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
75192024-11-14T04:51:45.523ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
75202024-11-14T04:51:45.523ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 204.854383ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
75212024-11-14T04:51:45.570ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 252.634406ms iteration = 14 status = {"physical_disks_added":0}
75222024-11-14T04:51:45.605ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75232024-11-14T04:51:45.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75242024-11-14T04:51:45.607ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75252024-11-14T04:51:45.609ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
75262024-11-14T04:51:45.609ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75272024-11-14T04:51:45.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75282024-11-14T04:51:45.611ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75292024-11-14T04:51:45.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
75302024-11-14T04:51:45.613ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75312024-11-14T04:51:45.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75322024-11-14T04:51:45.634ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
75332024-11-14T04:51:45.634ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
75342024-11-14T04:51:45.635ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a69fe25-ae3e-4b91-87da-793c5e050676", "content-length": "372", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75352024-11-14T04:51:45.635ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75362024-11-14T04:51:45.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75372024-11-14T04:51:45.681ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
75382024-11-14T04:51:45.681ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
75392024-11-14T04:51:45.681ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c1dbfd45-7b1a-48b6-80e4-e8ca9a1f3053", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75402024-11-14T04:51:45.681ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75412024-11-14T04:51:45.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75422024-11-14T04:51:45.683ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75432024-11-14T04:51:45.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75442024-11-14T04:51:45.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75452024-11-14T04:51:45.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75462024-11-14T04:51:45.689ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
75472024-11-14T04:51:45.689ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "861e698b-a1f3-4b7b-98bc-b99d6de8fa41", "content-length": "691", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75482024-11-14T04:51:45.689ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
75492024-11-14T04:51:45.689ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
75502024-11-14T04:51:45.689ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
75512024-11-14T04:51:45.691ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 373.662421ms iteration = 12 status = {}
75522024-11-14T04:51:45.694ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
75532024-11-14T04:51:45.694ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
75542024-11-14T04:51:45.694ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "504fda00-dc8f-411b-8079-67b85e33aa8f", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75552024-11-14T04:51:45.696ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 377.894813ms iteration = 13 status = {}
75562024-11-14T04:51:45.706ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75572024-11-14T04:51:45.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75582024-11-14T04:51:45.708ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
75592024-11-14T04:51:45.708ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75602024-11-14T04:51:45.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75612024-11-14T04:51:45.717ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
75622024-11-14T04:51:45.717ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1db84983-0a01-4211-b1f8-e6d64c779b3c", "content-length": "134", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75632024-11-14T04:51:45.717ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
75642024-11-14T04:51:45.717ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
75652024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
75662024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
75672024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
75682024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
75692024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
75702024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
75712024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
75722024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
75732024-11-14T04:51:45.718ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
75742024-11-14T04:51:45.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75752024-11-14T04:51:45.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75762024-11-14T04:51:45.720ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
75772024-11-14T04:51:45.720ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e37a7460-19d5-4237-b43b-6a21e0a641b9", "content-length": "2", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75782024-11-14T04:51:45.720ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
75792024-11-14T04:51:45.720ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "96ed10b2-32de-4446-b270-4c4c71d846d9", "content-length": "51", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75802024-11-14T04:51:45.720ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
75812024-11-14T04:51:45.720ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
75822024-11-14T04:51:45.882ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "be49032a-d96f-44cf-9f5d-52bc32398e83", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75832024-11-14T04:51:45.882ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
75842024-11-14T04:51:45.882ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
75852024-11-14T04:51:45.886ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "14ef0eb5-e424-4325-9956-4eaebb06f4d9", "content-length": "682", "date": "Thu, 14 Nov 2024 04:51:45 GMT"} })
75862024-11-14T04:51:45.886ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75872024-11-14T04:51:45.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75882024-11-14T04:51:45.890ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75892024-11-14T04:51:45.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75902024-11-14T04:51:45.893ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75912024-11-14T04:51:45.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75922024-11-14T04:51:45.896ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75932024-11-14T04:51:45.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75942024-11-14T04:51:45.903ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 585.542266ms iteration = 12 status = {}
75952024-11-14T04:51:49.706ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:53397
75962024-11-14T04:51:49.706ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53397 req_id = 2d54f5bf-8dbd-493a-bb9e-54a42e7c0fe3 roles = RoleSet { roles: {} } uri = /metrics/producers
75972024-11-14T04:51:49.706ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53397 req_id = 2d54f5bf-8dbd-493a-bb9e-54a42e7c0fe3 resource = Database result = Ok(()) uri = /metrics/producers
75982024-11-14T04:51:49.751ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = e5586e29-e301-4b24-9a2c-39545da75716
75992024-11-14T04:51:49.751ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44945 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53397 req_id = 2d54f5bf-8dbd-493a-bb9e-54a42e7c0fe3 response_code = 201 uri = /metrics/producers
76002024-11-14T04:51:50.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1165 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 23b45c0c-ff87-489b-a37d-99a92af926a6 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
76012024-11-14T04:51:55.252ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:53964
76022024-11-14T04:51:55.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = f8f87d95-252b-4b88-9a1f-9a37b50fd06c roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
76032024-11-14T04:51:55.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = f8f87d95-252b-4b88-9a1f-9a37b50fd06c resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
76042024-11-14T04:51:55.256ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3498 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = f8f87d95-252b-4b88-9a1f-9a37b50fd06c response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
76052024-11-14T04:51:55.256ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = fc2b1b74-9ea7-44ab-895b-2db5bcdd2f41 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
76062024-11-14T04:51:55.256ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = fc2b1b74-9ea7-44ab-895b-2db5bcdd2f41 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
76072024-11-14T04:51:55.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2776 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = fc2b1b74-9ea7-44ab-895b-2db5bcdd2f41 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
76082024-11-14T04:51:55.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
76092024-11-14T04:51:55.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
76102024-11-14T04:51:55.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76112024-11-14T04:51:55.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76122024-11-14T04:51:55.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76132024-11-14T04:51:55.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76142024-11-14T04:51:55.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76152024-11-14T04:51:55.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76162024-11-14T04:51:55.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76172024-11-14T04:51:55.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76182024-11-14T04:51:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76192024-11-14T04:51:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76202024-11-14T04:51:55.347ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76212024-11-14T04:51:55.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76222024-11-14T04:51:55.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76232024-11-14T04:51:55.363ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
76242024-11-14T04:51:55.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76252024-11-14T04:51:55.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76262024-11-14T04:51:55.376ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
76272024-11-14T04:51:55.376ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.636159ms iteration = 35 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
76282024-11-14T04:51:55.381ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.156549ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
76292024-11-14T04:52:00.682ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:64085
76302024-11-14T04:52:00.682ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64085 req_id = 66d5ba5b-9027-4d9a-bb9b-7e058e4ae601 roles = RoleSet { roles: {} } uri = /metrics/producers
76312024-11-14T04:52:00.682ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64085 req_id = 66d5ba5b-9027-4d9a-bb9b-7e058e4ae601 resource = Database result = Ok(()) uri = /metrics/producers
76322024-11-14T04:52:00.770ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = f4c8f330-91d6-4976-b637-c72f900d8c87
76332024-11-14T04:52:00.770ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88598 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64085 req_id = 66d5ba5b-9027-4d9a-bb9b-7e058e4ae601 response_code = 201 uri = /metrics/producers
76342024-11-14T04:52:02.604ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:64704
76352024-11-14T04:52:02.604ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64704 req_id = dad9167c-068e-4b2c-bc23-892239f83e89 roles = RoleSet { roles: {} } uri = /metrics/producers
76362024-11-14T04:52:02.604ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64704 req_id = dad9167c-068e-4b2c-bc23-892239f83e89 resource = Database result = Ok(()) uri = /metrics/producers
76372024-11-14T04:52:02.661ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 18d4ee8d-1d67-44aa-96f9-89dfcadba34d
76382024-11-14T04:52:02.661ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57873 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64704 req_id = dad9167c-068e-4b2c-bc23-892239f83e89 response_code = 201 uri = /metrics/producers
76392024-11-14T04:52:05.261ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1049 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 48b09b06-9ae2-433b-9df1-e662d6c50449 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
76402024-11-14T04:52:05.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
76412024-11-14T04:52:05.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
76422024-11-14T04:52:05.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76432024-11-14T04:52:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76442024-11-14T04:52:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76452024-11-14T04:52:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76462024-11-14T04:52:05.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76472024-11-14T04:52:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76482024-11-14T04:52:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76492024-11-14T04:52:05.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.13702ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
76502024-11-14T04:52:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76512024-11-14T04:52:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76522024-11-14T04:52:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76532024-11-14T04:52:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76542024-11-14T04:52:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76552024-11-14T04:52:05.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76562024-11-14T04:52:05.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
76572024-11-14T04:52:05.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76582024-11-14T04:52:05.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76592024-11-14T04:52:05.339ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
76602024-11-14T04:52:05.339ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.42938ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
76612024-11-14T04:52:10.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 15ab3008-4b18-4815-930f-a046c26352df roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
76622024-11-14T04:52:10.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 15ab3008-4b18-4815-930f-a046c26352df resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
76632024-11-14T04:52:10.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2728 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 15ab3008-4b18-4815-930f-a046c26352df response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
76642024-11-14T04:52:10.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 137e318a-ff41-444f-9b54-6f26393558e0 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
76652024-11-14T04:52:10.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 137e318a-ff41-444f-9b54-6f26393558e0 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
76662024-11-14T04:52:10.258ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2396 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 137e318a-ff41-444f-9b54-6f26393558e0 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
76672024-11-14T04:52:14.440ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:33790
76682024-11-14T04:52:14.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33790 req_id = fb1503c4-b781-41d3-90b3-fb1c6f8bfc7e roles = RoleSet { roles: {} } uri = /metrics/producers
76692024-11-14T04:52:14.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33790 req_id = fb1503c4-b781-41d3-90b3-fb1c6f8bfc7e resource = Database result = Ok(()) uri = /metrics/producers
76702024-11-14T04:52:14.459ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = bc37b1eb-3677-438e-a276-b6d3e6fe3f8e
76712024-11-14T04:52:14.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 19475 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33790 req_id = fb1503c4-b781-41d3-90b3-fb1c6f8bfc7e response_code = 201 uri = /metrics/producers
76722024-11-14T04:52:15.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
76732024-11-14T04:52:15.317ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
76742024-11-14T04:52:15.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
76752024-11-14T04:52:15.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76762024-11-14T04:52:15.317ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
76772024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76782024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.93\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
76792024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
76802024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76812024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
76822024-11-14T04:52:15.318ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
76832024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
76842024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76852024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
76862024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
76872024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
76882024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76892024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
76902024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
76912024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
76922024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
76932024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
76942024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76952024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
76962024-11-14T04:52:15.318ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
76972024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
76982024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
76992024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
77002024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
77012024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
77022024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
77032024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
77042024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
77052024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
77062024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
77072024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
77082024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
77092024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
77102024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
77112024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
77122024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
77132024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77142024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
77152024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
77162024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
77172024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
77182024-11-14T04:52:15.318ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:42:15.318250323 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
77192024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
77202024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
77212024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
77222024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
77232024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
77242024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77252024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77262024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77272024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77282024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
77292024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
77302024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
77312024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77322024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77332024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
77342024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
77352024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
77362024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
77372024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
77382024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77392024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
77402024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
77412024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
77422024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
77432024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77442024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
77452024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
77462024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
77472024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
77482024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
77492024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
77502024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
77512024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
77522024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
77532024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
77542024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
77552024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
77562024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
77572024-11-14T04:52:15.318ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
77582024-11-14T04:52:15.318ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1314, tv_nsec: 93031778 } })
77592024-11-14T04:52:15.319ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1314, tv_nsec: 93055949 } })
77602024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
77612024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
77622024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
77632024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
77642024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
77652024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77662024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
77672024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
77682024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
77692024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.335429ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
77702024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
77712024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
77722024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
77732024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.636747ms iteration = 8 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
77742024-11-14T04:52:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
77752024-11-14T04:52:15.322ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77762024-11-14T04:52:15.323ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77772024-11-14T04:52:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77782024-11-14T04:52:15.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77792024-11-14T04:52:15.324ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77802024-11-14T04:52:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
77812024-11-14T04:52:15.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77822024-11-14T04:52:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77832024-11-14T04:52:15.325ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
77842024-11-14T04:52:15.326ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77852024-11-14T04:52:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
77862024-11-14T04:52:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
77872024-11-14T04:52:15.327ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77882024-11-14T04:52:15.327ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.011861ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
77892024-11-14T04:52:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77902024-11-14T04:52:15.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77912024-11-14T04:52:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
77922024-11-14T04:52:15.328ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
77932024-11-14T04:52:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
77942024-11-14T04:52:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
77952024-11-14T04:52:15.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77962024-11-14T04:52:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77972024-11-14T04:52:15.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
77982024-11-14T04:52:15.330ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77992024-11-14T04:52:15.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78002024-11-14T04:52:15.333ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78012024-11-14T04:52:15.333ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78022024-11-14T04:52:15.333ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78032024-11-14T04:52:15.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
78042024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78052024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78062024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
78072024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
78082024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
78092024-11-14T04:52:15.339ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
78102024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
78112024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
78122024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
78132024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
78142024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
78152024-11-14T04:52:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
78162024-11-14T04:52:15.341ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
78172024-11-14T04:52:15.341ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78182024-11-14T04:52:15.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
78192024-11-14T04:52:15.341ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1314, tv_nsec: 115892370 } })
78202024-11-14T04:52:15.342ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
78212024-11-14T04:52:15.342ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
78222024-11-14T04:52:15.343ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1314, tv_nsec: 117176427 } })
78232024-11-14T04:52:15.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
78242024-11-14T04:52:15.343ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:42:15.318250323 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
78252024-11-14T04:52:15.343ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 24.915661ms iteration = 7 status = {"expiration":"2024-11-14T04:42:15.318250323Z","pruned":[]}
78262024-11-14T04:52:15.343ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
78272024-11-14T04:52:15.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
78282024-11-14T04:52:15.344ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
78292024-11-14T04:52:15.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
78302024-11-14T04:52:15.349ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78312024-11-14T04:52:15.349ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 30.242245ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
78322024-11-14T04:52:15.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
78332024-11-14T04:52:15.353ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
78342024-11-14T04:52:15.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
78352024-11-14T04:52:15.361ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 42.383317ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
78362024-11-14T04:52:15.362ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78372024-11-14T04:52:15.362ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
78382024-11-14T04:52:15.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
78392024-11-14T04:52:15.364ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
78402024-11-14T04:52:15.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
78412024-11-14T04:52:15.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
78422024-11-14T04:52:15.365ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
78432024-11-14T04:52:15.365ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
78442024-11-14T04:52:15.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
78452024-11-14T04:52:15.365ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78462024-11-14T04:52:15.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
78472024-11-14T04:52:15.367ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78482024-11-14T04:52:15.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
78492024-11-14T04:52:15.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
78502024-11-14T04:52:15.368ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
78512024-11-14T04:52:15.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
78522024-11-14T04:52:15.369ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78532024-11-14T04:52:15.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
78542024-11-14T04:52:15.371ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
78552024-11-14T04:52:15.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
78562024-11-14T04:52:15.374ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78572024-11-14T04:52:15.376ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
78582024-11-14T04:52:15.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78592024-11-14T04:52:15.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
78602024-11-14T04:52:15.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78612024-11-14T04:52:15.377ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1314, tv_nsec: 151727837 } })
78622024-11-14T04:52:15.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
78632024-11-14T04:52:15.377ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
78642024-11-14T04:52:15.377ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
78652024-11-14T04:52:15.377ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
78662024-11-14T04:52:15.377ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
78672024-11-14T04:52:15.378ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
78682024-11-14T04:52:15.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78692024-11-14T04:52:15.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
78702024-11-14T04:52:15.386ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 68.447922ms iteration = 4 status = {"nsuccess":5}
78712024-11-14T04:52:15.387ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78722024-11-14T04:52:15.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
78732024-11-14T04:52:15.391ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
78742024-11-14T04:52:15.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
78752024-11-14T04:52:15.412ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 94.484137ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
78762024-11-14T04:52:15.425ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 106.38021ms iteration = 13 status = {"errors":[],"records_set_to_done":[]}
78772024-11-14T04:52:15.428ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78782024-11-14T04:52:15.429ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
78792024-11-14T04:52:15.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
78802024-11-14T04:52:15.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
78812024-11-14T04:52:15.435ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
78822024-11-14T04:52:15.436ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
78832024-11-14T04:52:15.436ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 117.124432ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
78842024-11-14T04:52:15.436ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 118.193963ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
78852024-11-14T04:52:15.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
78862024-11-14T04:52:15.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
78872024-11-14T04:52:15.449ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
78882024-11-14T04:52:15.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78892024-11-14T04:52:15.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
78902024-11-14T04:52:15.462ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
78912024-11-14T04:52:15.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
78922024-11-14T04:52:15.465ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
78932024-11-14T04:52:15.465ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
78942024-11-14T04:52:15.465ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 147.141651ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
78952024-11-14T04:52:15.470ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 152.098375ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
78962024-11-14T04:52:15.471ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
78972024-11-14T04:52:15.471ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 153.054732ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
78982024-11-14T04:52:15.472ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
78992024-11-14T04:52:15.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
79002024-11-14T04:52:15.476ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
79012024-11-14T04:52:15.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
79022024-11-14T04:52:15.482ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79032024-11-14T04:52:15.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
79042024-11-14T04:52:15.488ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
79052024-11-14T04:52:15.488ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79062024-11-14T04:52:15.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
79072024-11-14T04:52:15.490ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 173.087513ms iteration = 13 status = 0
79082024-11-14T04:52:15.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
79092024-11-14T04:52:15.491ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
79102024-11-14T04:52:15.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
79112024-11-14T04:52:15.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79122024-11-14T04:52:15.494ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
79132024-11-14T04:52:15.494ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
79142024-11-14T04:52:15.494ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 176.540123ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
79152024-11-14T04:52:15.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
79162024-11-14T04:52:15.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79172024-11-14T04:52:15.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79182024-11-14T04:52:15.501ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79192024-11-14T04:52:15.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
79202024-11-14T04:52:15.503ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79212024-11-14T04:52:15.505ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
79222024-11-14T04:52:15.505ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
79232024-11-14T04:52:15.505ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
79242024-11-14T04:52:15.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79252024-11-14T04:52:15.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79262024-11-14T04:52:15.506ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
79272024-11-14T04:52:15.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79282024-11-14T04:52:15.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
79292024-11-14T04:52:15.508ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 189.459967ms iteration = 13 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
79302024-11-14T04:52:15.520ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
79312024-11-14T04:52:15.520ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 202.953557ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
79322024-11-14T04:52:15.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
79332024-11-14T04:52:15.525ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
79342024-11-14T04:52:15.525ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
79352024-11-14T04:52:15.525ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
79362024-11-14T04:52:15.525ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79372024-11-14T04:52:15.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79382024-11-14T04:52:15.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79392024-11-14T04:52:15.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
79402024-11-14T04:52:15.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79412024-11-14T04:52:15.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79422024-11-14T04:52:15.558ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 240.825584ms iteration = 15 status = {"physical_disks_added":0}
79432024-11-14T04:52:15.567ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79442024-11-14T04:52:15.571ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
79452024-11-14T04:52:15.571ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79462024-11-14T04:52:15.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79472024-11-14T04:52:15.583ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
79482024-11-14T04:52:15.583ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 265.777487ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
79492024-11-14T04:52:15.586ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
79502024-11-14T04:52:15.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
79512024-11-14T04:52:15.591ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
79522024-11-14T04:52:15.591ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
79532024-11-14T04:52:15.591ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79542024-11-14T04:52:15.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79552024-11-14T04:52:15.600ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
79562024-11-14T04:52:15.600ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 281.982666ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
79572024-11-14T04:52:15.616ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
79582024-11-14T04:52:15.616ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
79592024-11-14T04:52:15.616ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1f5c508-9129-4a5f-95e1-f06f58558389", "content-length": "26", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
79602024-11-14T04:52:15.616ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
79612024-11-14T04:52:15.618ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
79622024-11-14T04:52:15.644ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
79632024-11-14T04:52:15.644ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "0ff8ceaf-85cf-4168-a1c7-e06b092ede54", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
79642024-11-14T04:52:15.644ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
79652024-11-14T04:52:15.646ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5d839314-451d-4565-9f18-799a874fef50", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
79662024-11-14T04:52:15.646ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
79672024-11-14T04:52:15.647ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
79682024-11-14T04:52:15.647ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 329.299196ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
79692024-11-14T04:52:15.713ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
79702024-11-14T04:52:15.713ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
79712024-11-14T04:52:15.713ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0592af4-50c3-4ba6-a1f9-bbc575b67b46", "content-length": "26", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
79722024-11-14T04:52:15.713ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
79732024-11-14T04:52:15.715ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
79742024-11-14T04:52:15.721ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
79752024-11-14T04:52:15.721ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
79762024-11-14T04:52:15.721ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "81b26ce1-7941-4500-a305-99fad9df4a18", "content-length": "372", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
79772024-11-14T04:52:15.721ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79782024-11-14T04:52:15.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79792024-11-14T04:52:15.724ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79802024-11-14T04:52:15.726ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
79812024-11-14T04:52:15.726ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79822024-11-14T04:52:15.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79832024-11-14T04:52:15.729ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
79842024-11-14T04:52:15.729ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
79852024-11-14T04:52:15.729ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
79862024-11-14T04:52:15.729ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 410.836228ms iteration = 8 status = {"generation":2}
79872024-11-14T04:52:15.752ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
79882024-11-14T04:52:15.752ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
79892024-11-14T04:52:15.752ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79902024-11-14T04:52:15.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79912024-11-14T04:52:15.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79922024-11-14T04:52:15.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
79932024-11-14T04:52:15.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79942024-11-14T04:52:15.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79952024-11-14T04:52:15.757ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79962024-11-14T04:52:15.759ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
79972024-11-14T04:52:15.759ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
79982024-11-14T04:52:15.759ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
79992024-11-14T04:52:15.759ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 441.459475ms iteration = 8 status = {"generation":1}
80002024-11-14T04:52:15.759ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
80012024-11-14T04:52:15.759ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
80022024-11-14T04:52:15.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
80032024-11-14T04:52:15.761ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80042024-11-14T04:52:15.761ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
80052024-11-14T04:52:15.761ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
80062024-11-14T04:52:15.762ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f13d74a2-7705-4f12-83a4-6118404172ce", "content-length": "26", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
80072024-11-14T04:52:15.762ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
80082024-11-14T04:52:15.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
80092024-11-14T04:52:15.762ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
80102024-11-14T04:52:15.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
80112024-11-14T04:52:15.764ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
80122024-11-14T04:52:15.765ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
80132024-11-14T04:52:15.765ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f0c6c48-aa24-4a07-a5b8-882229d4ff75", "content-length": "1", "date": "Thu, 14 Nov 2024 04:52:14 GMT"} })
80142024-11-14T04:52:15.767ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
80152024-11-14T04:52:15.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
80162024-11-14T04:52:15.767ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
80172024-11-14T04:52:15.767ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
80182024-11-14T04:52:15.767ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7561b011-4219-4120-b26d-417b93ab9ef1", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
80192024-11-14T04:52:15.767ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80202024-11-14T04:52:15.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80212024-11-14T04:52:15.769ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 451.108714ms iteration = 13 status = null
80222024-11-14T04:52:15.769ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80232024-11-14T04:52:15.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80242024-11-14T04:52:15.773ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80252024-11-14T04:52:15.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80262024-11-14T04:52:15.776ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
80272024-11-14T04:52:15.776ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
80282024-11-14T04:52:15.776ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ac4feeec-d700-49be-ae59-37715d857677", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
80292024-11-14T04:52:15.778ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 460.478565ms iteration = 14 status = {}
80302024-11-14T04:52:15.789ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
80312024-11-14T04:52:15.847ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 529.655977ms iteration = 13 status = {}
80322024-11-14T04:52:15.881ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
80332024-11-14T04:52:15.881ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "2f4ea9c4-b4f5-4a4e-a2ba-6f85c4fbd7b8", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
80342024-11-14T04:52:15.881ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
80352024-11-14T04:52:15.883ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "f39b16bb-098c-47e7-807c-9b09114df0a5", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
80362024-11-14T04:52:15.883ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
80372024-11-14T04:52:15.884ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5f6b9fdb-3d5f-43b5-bc70-dec717ace5d3", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
80382024-11-14T04:52:15.884ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
80392024-11-14T04:52:15.887ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
80402024-11-14T04:52:15.887ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 569.344387ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
80412024-11-14T04:52:15.902ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
80422024-11-14T04:52:15.928ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80432024-11-14T04:52:15.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80442024-11-14T04:52:15.930ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
80452024-11-14T04:52:15.930ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80462024-11-14T04:52:15.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80472024-11-14T04:52:15.940ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
80482024-11-14T04:52:15.940ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "95f1bff4-d13b-49c9-8a8d-c9a3fe4468d0", "content-length": "134", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
80492024-11-14T04:52:15.940ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
80502024-11-14T04:52:15.940ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
80512024-11-14T04:52:15.940ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
80522024-11-14T04:52:15.941ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
80532024-11-14T04:52:15.941ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
80542024-11-14T04:52:15.941ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
80552024-11-14T04:52:15.941ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
80562024-11-14T04:52:15.941ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
80572024-11-14T04:52:15.941ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
80582024-11-14T04:52:15.941ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
80592024-11-14T04:52:15.941ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
80602024-11-14T04:52:15.941ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80612024-11-14T04:52:15.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80622024-11-14T04:52:15.943ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
80632024-11-14T04:52:15.943ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b1d9047-a055-4b27-a21a-cb19b2aadc18", "content-length": "2", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
80642024-11-14T04:52:15.943ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
80652024-11-14T04:52:15.943ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a698846-bcb6-4dd9-b142-9fdf5c6b131b", "content-length": "51", "date": "Thu, 14 Nov 2024 04:52:15 GMT"} })
80662024-11-14T04:52:15.943ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
80672024-11-14T04:52:15.943ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
80682024-11-14T04:52:16.006ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8966d33a-8c4b-4789-b72e-57322b652331", "content-length": "691", "date": "Thu, 14 Nov 2024 04:52:16 GMT"} })
80692024-11-14T04:52:16.006ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
80702024-11-14T04:52:16.006ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
80712024-11-14T04:52:16.006ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
80722024-11-14T04:52:16.007ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 689.715863ms iteration = 13 status = {}
80732024-11-14T04:52:16.103ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "825aae5c-1419-425f-bbd4-9aab5d4aed6e", "date": "Thu, 14 Nov 2024 04:52:16 GMT"} })
80742024-11-14T04:52:16.104ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
80752024-11-14T04:52:16.104ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
80762024-11-14T04:52:16.108ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0eefd96-71cf-437c-8843-e991ff3f610c", "content-length": "682", "date": "Thu, 14 Nov 2024 04:52:16 GMT"} })
80772024-11-14T04:52:16.109ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80782024-11-14T04:52:16.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80792024-11-14T04:52:16.113ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80802024-11-14T04:52:16.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80812024-11-14T04:52:16.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80822024-11-14T04:52:16.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80832024-11-14T04:52:16.118ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80842024-11-14T04:52:16.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80852024-11-14T04:52:16.126ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 808.244715ms iteration = 13 status = {}
80862024-11-14T04:52:19.101ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:35282
80872024-11-14T04:52:19.102ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:35282 req_id = 0d1d22ad-663d-4642-961d-428638daf374 roles = RoleSet { roles: {} } uri = /metrics/producers
80882024-11-14T04:52:19.102ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:35282 req_id = 0d1d22ad-663d-4642-961d-428638daf374 resource = Database result = Ok(()) uri = /metrics/producers
80892024-11-14T04:52:19.153ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = c073d51c-abaa-48df-abb6-2f053e9c1bec
80902024-11-14T04:52:19.154ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52076 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:35282 req_id = 0d1d22ad-663d-4642-961d-428638daf374 response_code = 201 uri = /metrics/producers
80912024-11-14T04:52:20.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1190 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 860f2d86-faca-4f3a-be4b-52304f45ef19 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
80922024-11-14T04:52:25.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = eb6c1bf3-48d4-459a-bbd5-006cee7dfdc3 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
80932024-11-14T04:52:25.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = eb6c1bf3-48d4-459a-bbd5-006cee7dfdc3 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
80942024-11-14T04:52:25.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3083 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = eb6c1bf3-48d4-459a-bbd5-006cee7dfdc3 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
80952024-11-14T04:52:25.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = b8d12411-9327-4cc4-8797-a2650cce21a7 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
80962024-11-14T04:52:25.256ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = b8d12411-9327-4cc4-8797-a2650cce21a7 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
80972024-11-14T04:52:25.258ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2279 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = b8d12411-9327-4cc4-8797-a2650cce21a7 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
80982024-11-14T04:52:25.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
80992024-11-14T04:52:25.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
81002024-11-14T04:52:25.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81012024-11-14T04:52:25.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81022024-11-14T04:52:25.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81032024-11-14T04:52:25.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81042024-11-14T04:52:25.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81052024-11-14T04:52:25.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
81062024-11-14T04:52:25.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81072024-11-14T04:52:25.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81082024-11-14T04:52:25.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81092024-11-14T04:52:25.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81102024-11-14T04:52:25.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81112024-11-14T04:52:25.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
81122024-11-14T04:52:25.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81132024-11-14T04:52:25.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81142024-11-14T04:52:25.346ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
81152024-11-14T04:52:25.346ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.924488ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
81162024-11-14T04:52:25.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81172024-11-14T04:52:25.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81182024-11-14T04:52:25.384ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.210858ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
81192024-11-14T04:52:35.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1035 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 04717529-065a-4041-96e7-9ed6030cdfcc response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
81202024-11-14T04:52:35.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
81212024-11-14T04:52:35.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
81222024-11-14T04:52:35.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81232024-11-14T04:52:35.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81242024-11-14T04:52:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81252024-11-14T04:52:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81262024-11-14T04:52:35.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81272024-11-14T04:52:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
81282024-11-14T04:52:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81292024-11-14T04:52:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81302024-11-14T04:52:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81312024-11-14T04:52:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81322024-11-14T04:52:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81332024-11-14T04:52:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
81342024-11-14T04:52:35.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81352024-11-14T04:52:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81362024-11-14T04:52:35.339ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
81372024-11-14T04:52:35.339ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.303926ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
81382024-11-14T04:52:35.347ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81392024-11-14T04:52:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81402024-11-14T04:52:35.375ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.681511ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
81412024-11-14T04:52:41.665ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:52409
81422024-11-14T04:52:41.665ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52409 req_id = 052fb2f5-f2b7-47d8-83c0-90b8eab1551b roles = RoleSet { roles: {} } uri = /metrics/producers
81432024-11-14T04:52:41.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52409 req_id = 052fb2f5-f2b7-47d8-83c0-90b8eab1551b resource = Database result = Ok(()) uri = /metrics/producers
81442024-11-14T04:52:41.681ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = b584ea61-acb3-47a3-a173-0d03f6cae9c9
81452024-11-14T04:52:41.681ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 16534 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52409 req_id = 052fb2f5-f2b7-47d8-83c0-90b8eab1551b response_code = 201 uri = /metrics/producers
81462024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
81472024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
81482024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
81492024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
81502024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81512024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81522024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
81532024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
81542024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
81552024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
81562024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
81572024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81582024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
81592024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
81602024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
81612024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
81622024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
81632024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81642024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
81652024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81662024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81672024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
81682024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
81692024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
81702024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
81712024-11-14T04:52:45.318ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
81722024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81732024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
81742024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81752024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81762024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
81772024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81782024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81792024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
81802024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81812024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81822024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
81832024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81842024-11-14T04:52:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
81852024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
81862024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
81872024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
81882024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
81892024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
81902024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
81912024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
81922024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
81932024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81942024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
81952024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
81962024-11-14T04:52:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
81972024-11-14T04:52:45.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81982024-11-14T04:52:45.321ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1344, tv_nsec: 95847934 } })
81992024-11-14T04:52:45.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82002024-11-14T04:52:45.321ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.45312ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
82012024-11-14T04:52:45.321ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
82022024-11-14T04:52:45.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82032024-11-14T04:52:45.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
82042024-11-14T04:52:45.323ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82052024-11-14T04:52:45.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82062024-11-14T04:52:45.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
82072024-11-14T04:52:45.324ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
82082024-11-14T04:52:45.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
82092024-11-14T04:52:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
82102024-11-14T04:52:45.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82112024-11-14T04:52:45.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82122024-11-14T04:52:45.326ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82132024-11-14T04:52:45.327ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82142024-11-14T04:52:45.328ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1344, tv_nsec: 102421564 } })
82152024-11-14T04:52:45.328ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
82162024-11-14T04:52:45.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
82172024-11-14T04:52:45.328ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82182024-11-14T04:52:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82192024-11-14T04:52:45.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
82202024-11-14T04:52:45.330ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
82212024-11-14T04:52:45.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
82222024-11-14T04:52:45.336ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82232024-11-14T04:52:45.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82242024-11-14T04:52:45.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82252024-11-14T04:52:45.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82262024-11-14T04:52:45.353ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
82272024-11-14T04:52:45.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
82282024-11-14T04:52:45.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82292024-11-14T04:52:45.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
82302024-11-14T04:52:45.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82312024-11-14T04:52:45.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82322024-11-14T04:52:45.380ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
82332024-11-14T04:52:45.380ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82342024-11-14T04:52:45.381ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e85cf3df-49dc-4be4-a4d0-fb6f8bc4a6a0", "content-length": "26", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
82352024-11-14T04:52:45.381ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82362024-11-14T04:52:45.382ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
82372024-11-14T04:52:45.382ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82382024-11-14T04:52:45.382ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "af6d6e0d-97e0-4123-b55b-1cc208117b4b", "content-length": "26", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
82392024-11-14T04:52:45.382ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82402024-11-14T04:52:45.384ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
82412024-11-14T04:52:45.384ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
82422024-11-14T04:52:45.385ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
82432024-11-14T04:52:45.386ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.690501ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
82442024-11-14T04:52:45.386ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82452024-11-14T04:52:45.387ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82462024-11-14T04:52:45.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82472024-11-14T04:52:45.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
82482024-11-14T04:52:45.389ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82492024-11-14T04:52:45.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82502024-11-14T04:52:45.392ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82512024-11-14T04:52:45.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82522024-11-14T04:52:45.392ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1344, tv_nsec: 167101347 } })
82532024-11-14T04:52:45.392ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82542024-11-14T04:52:45.392ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
82552024-11-14T04:52:45.392ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
82562024-11-14T04:52:45.393ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
82572024-11-14T04:52:45.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
82582024-11-14T04:52:45.399ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
82592024-11-14T04:52:45.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
82602024-11-14T04:52:45.404ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
82612024-11-14T04:52:45.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
82622024-11-14T04:52:45.407ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82632024-11-14T04:52:45.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 91.036406ms iteration = 14 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
82642024-11-14T04:52:45.410ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82652024-11-14T04:52:45.410ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
82662024-11-14T04:52:45.410ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
82672024-11-14T04:52:45.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 92.21086ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
82682024-11-14T04:52:45.410ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
82692024-11-14T04:52:45.410ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
82702024-11-14T04:52:45.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
82712024-11-14T04:52:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
82722024-11-14T04:52:45.414ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82732024-11-14T04:52:45.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82742024-11-14T04:52:45.416ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82752024-11-14T04:52:45.418ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
82762024-11-14T04:52:45.418ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82772024-11-14T04:52:45.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82782024-11-14T04:52:45.418ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 99.219523ms iteration = 14 status = {"errors":[],"records_set_to_done":[]}
82792024-11-14T04:52:45.419ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 100.024296ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
82802024-11-14T04:52:45.423ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
82812024-11-14T04:52:45.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
82822024-11-14T04:52:45.430ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
82832024-11-14T04:52:45.430ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 112.171898ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
82842024-11-14T04:52:45.438ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 120.614483ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
82852024-11-14T04:52:45.451ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82862024-11-14T04:52:45.451ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
82872024-11-14T04:52:45.451ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 132.96649ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
82882024-11-14T04:52:45.455ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82892024-11-14T04:52:45.457ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
82902024-11-14T04:52:45.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
82912024-11-14T04:52:45.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
82922024-11-14T04:52:45.458ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82932024-11-14T04:52:45.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82942024-11-14T04:52:45.461ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 143.100614ms iteration = 14 status = 0
82952024-11-14T04:52:45.465ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 147.601404ms iteration = 14 status = {}
82962024-11-14T04:52:45.473ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
82972024-11-14T04:52:45.473ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82982024-11-14T04:52:45.473ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "61d3a2d1-b1c6-4c5e-8185-d18ecd49956a", "content-length": "26", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
82992024-11-14T04:52:45.473ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83002024-11-14T04:52:45.475ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
83012024-11-14T04:52:45.481ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
83022024-11-14T04:52:45.481ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "624f35a2-0808-4090-8ea8-5d68a563f98a", "content-length": "1", "date": "Thu, 14 Nov 2024 04:52:44 GMT"} })
83032024-11-14T04:52:45.483ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
83042024-11-14T04:52:45.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
83052024-11-14T04:52:45.485ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 167.735537ms iteration = 14 status = null
83062024-11-14T04:52:45.499ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83072024-11-14T04:52:45.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
83082024-11-14T04:52:45.504ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 185.020498ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
83092024-11-14T04:52:45.539ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 221.586987ms iteration = 16 status = {"physical_disks_added":0}
83102024-11-14T04:52:45.571ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
83112024-11-14T04:52:45.571ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
83122024-11-14T04:52:45.572ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "94c32390-d739-4348-a558-4c669d7b7822", "content-length": "372", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83132024-11-14T04:52:45.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83142024-11-14T04:52:45.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83152024-11-14T04:52:45.600ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83162024-11-14T04:52:45.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
83172024-11-14T04:52:45.621ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83182024-11-14T04:52:45.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83192024-11-14T04:52:45.622ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
83202024-11-14T04:52:45.622ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83212024-11-14T04:52:45.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83222024-11-14T04:52:45.626ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
83232024-11-14T04:52:45.626ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
83242024-11-14T04:52:45.627ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "769ea59f-ab72-4854-8877-09e4d271b3ae", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83252024-11-14T04:52:45.627ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83262024-11-14T04:52:45.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83272024-11-14T04:52:45.630ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83282024-11-14T04:52:45.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83292024-11-14T04:52:45.634ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83302024-11-14T04:52:45.634ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83312024-11-14T04:52:45.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83322024-11-14T04:52:45.636ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
83332024-11-14T04:52:45.636ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
83342024-11-14T04:52:45.636ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83352024-11-14T04:52:45.636ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb9a6c33-e336-471b-8e64-0cd6987ac503", "content-length": "134", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83362024-11-14T04:52:45.636ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
83372024-11-14T04:52:45.636ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
83382024-11-14T04:52:45.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
83392024-11-14T04:52:45.636ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
83402024-11-14T04:52:45.636ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
83412024-11-14T04:52:45.636ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83422024-11-14T04:52:45.636ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83432024-11-14T04:52:45.636ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83442024-11-14T04:52:45.636ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
83452024-11-14T04:52:45.637ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83462024-11-14T04:52:45.637ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
83472024-11-14T04:52:45.637ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
83482024-11-14T04:52:45.637ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83492024-11-14T04:52:45.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83502024-11-14T04:52:45.638ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83512024-11-14T04:52:45.638ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
83522024-11-14T04:52:45.638ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
83532024-11-14T04:52:45.639ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3bd4ea68-9644-43d0-bf66-4c3a1c488354", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83542024-11-14T04:52:45.639ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
83552024-11-14T04:52:45.639ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "00e65feb-d7f6-409e-a43d-a029206a8915", "content-length": "2", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83562024-11-14T04:52:45.639ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
83572024-11-14T04:52:45.639ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "2128f990-ece8-4206-8c72-a01e531d27ab", "content-length": "51", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83582024-11-14T04:52:45.639ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
83592024-11-14T04:52:45.639ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
83602024-11-14T04:52:45.639ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
83612024-11-14T04:52:45.640ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83622024-11-14T04:52:45.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
83632024-11-14T04:52:45.640ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 322.501019ms iteration = 15 status = {}
83642024-11-14T04:52:45.735ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
83652024-11-14T04:52:45.743ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2080517e-ba5e-47e4-b1c9-74d189d3fb59", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83662024-11-14T04:52:45.743ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
83672024-11-14T04:52:45.743ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
83682024-11-14T04:52:45.744ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c61d291f-2bd5-4c27-8e9f-e7cecc681b50", "content-length": "682", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83692024-11-14T04:52:45.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83702024-11-14T04:52:45.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83712024-11-14T04:52:45.744ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "de39fa44-b252-4b7b-804c-a992ff1717eb", "content-length": "691", "date": "Thu, 14 Nov 2024 04:52:45 GMT"} })
83722024-11-14T04:52:45.745ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
83732024-11-14T04:52:45.745ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
83742024-11-14T04:52:45.745ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
83752024-11-14T04:52:45.745ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 427.709296ms iteration = 14 status = {}
83762024-11-14T04:52:45.748ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83772024-11-14T04:52:45.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83782024-11-14T04:52:45.751ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83792024-11-14T04:52:45.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83802024-11-14T04:52:45.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83812024-11-14T04:52:45.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83822024-11-14T04:52:45.760ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 441.738402ms iteration = 14 status = {}
83832024-11-14T04:52:50.259ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1232 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = afac1f86-ce42-4975-a0b3-0aef3a028e9e response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
83842024-11-14T04:52:55.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 0c657bea-2ce0-437e-a5fb-f10b20a9ab37 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
83852024-11-14T04:52:55.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 0c657bea-2ce0-437e-a5fb-f10b20a9ab37 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
83862024-11-14T04:52:55.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2651 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 0c657bea-2ce0-437e-a5fb-f10b20a9ab37 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
83872024-11-14T04:52:55.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 97a70f36-a0e0-401d-b7da-dcfca1a4ee94 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
83882024-11-14T04:52:55.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 97a70f36-a0e0-401d-b7da-dcfca1a4ee94 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
83892024-11-14T04:52:55.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2238 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 97a70f36-a0e0-401d-b7da-dcfca1a4ee94 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
83902024-11-14T04:52:55.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
83912024-11-14T04:52:55.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
83922024-11-14T04:52:55.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83932024-11-14T04:52:55.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83942024-11-14T04:52:55.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83952024-11-14T04:52:55.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83962024-11-14T04:52:55.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83972024-11-14T04:52:55.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
83982024-11-14T04:52:55.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83992024-11-14T04:52:55.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84002024-11-14T04:52:55.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84012024-11-14T04:52:55.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84022024-11-14T04:52:55.352ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84032024-11-14T04:52:55.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84042024-11-14T04:52:55.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84052024-11-14T04:52:55.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
84062024-11-14T04:52:55.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84072024-11-14T04:52:55.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84082024-11-14T04:52:55.372ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
84092024-11-14T04:52:55.372ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.795966ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
84102024-11-14T04:52:55.379ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 61.550522ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
84112024-11-14T04:53:05.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1259 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = b4dd29e9-ccf9-4376-9852-aca4297aeaff response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
84122024-11-14T04:53:05.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
84132024-11-14T04:53:05.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84142024-11-14T04:53:05.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
84152024-11-14T04:53:05.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84162024-11-14T04:53:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84172024-11-14T04:53:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84182024-11-14T04:53:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84192024-11-14T04:53:05.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84202024-11-14T04:53:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84212024-11-14T04:53:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
84222024-11-14T04:53:05.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84232024-11-14T04:53:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84242024-11-14T04:53:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84252024-11-14T04:53:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84262024-11-14T04:53:05.348ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.532203ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
84272024-11-14T04:53:05.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84282024-11-14T04:53:05.354ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
84292024-11-14T04:53:05.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84302024-11-14T04:53:05.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84312024-11-14T04:53:05.363ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
84322024-11-14T04:53:05.363ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.379493ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
84332024-11-14T04:53:10.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 22bbeeea-3e22-40c5-bc2a-8db5d1a7e249 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
84342024-11-14T04:53:10.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 22bbeeea-3e22-40c5-bc2a-8db5d1a7e249 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
84352024-11-14T04:53:10.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2922 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 22bbeeea-3e22-40c5-bc2a-8db5d1a7e249 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
84362024-11-14T04:53:10.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 5bb4c249-c55d-4476-81c3-a8589c383d72 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
84372024-11-14T04:53:10.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 5bb4c249-c55d-4476-81c3-a8589c383d72 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
84382024-11-14T04:53:10.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2603 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 5bb4c249-c55d-4476-81c3-a8589c383d72 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
84392024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
84402024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
84412024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
84422024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84432024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
84442024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
84452024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
84462024-11-14T04:53:15.317ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:43:15.317302630 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
84472024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
84482024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
84492024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
84502024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
84512024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
84522024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
84532024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
84542024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
84552024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
84562024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
84572024-11-14T04:53:15.317ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
84582024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
84592024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
84602024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
84612024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
84622024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
84632024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
84642024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
84652024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84662024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
84672024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
84682024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
84692024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
84702024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
84712024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84722024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
84732024-11-14T04:53:15.317ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
84742024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
84752024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84762024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
84772024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
84782024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
84792024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84802024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
84812024-11-14T04:53:15.317ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
84822024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.81\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
84832024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
84842024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
84852024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
84862024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
84872024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
84882024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
84892024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
84902024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
84912024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84922024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
84932024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
84942024-11-14T04:53:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
84952024-11-14T04:53:15.318ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1374, tv_nsec: 92621676 } })
84962024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84972024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
84982024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
84992024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85002024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
85012024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85022024-11-14T04:53:15.318ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1374, tv_nsec: 92685578 } })
85032024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
85042024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
85052024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
85062024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
85072024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
85082024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
85092024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
85102024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
85112024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
85122024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
85132024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
85142024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
85152024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
85162024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
85172024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
85182024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
85192024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
85202024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
85212024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
85222024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
85232024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
85242024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
85252024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
85262024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
85272024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.249246ms iteration = 9 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
85282024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
85292024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.305568ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
85302024-11-14T04:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
85312024-11-14T04:53:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
85322024-11-14T04:53:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
85332024-11-14T04:53:15.320ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:43:15.317302630 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
85342024-11-14T04:53:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.08881ms iteration = 8 status = {"expiration":"2024-11-14T04:43:15.317302630Z","pruned":[]}
85352024-11-14T04:53:15.321ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.867223ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
85362024-11-14T04:53:15.322ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85372024-11-14T04:53:15.323ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85382024-11-14T04:53:15.323ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85392024-11-14T04:53:15.324ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1374, tv_nsec: 99033622 } })
85402024-11-14T04:53:15.324ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
85412024-11-14T04:53:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
85422024-11-14T04:53:15.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85432024-11-14T04:53:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85442024-11-14T04:53:15.325ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1374, tv_nsec: 99868776 } })
85452024-11-14T04:53:15.325ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
85462024-11-14T04:53:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
85472024-11-14T04:53:15.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85482024-11-14T04:53:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85492024-11-14T04:53:15.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
85502024-11-14T04:53:15.326ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85512024-11-14T04:53:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85522024-11-14T04:53:15.327ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
85532024-11-14T04:53:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
85542024-11-14T04:53:15.333ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 15.337324ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
85552024-11-14T04:53:15.347ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85562024-11-14T04:53:15.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85572024-11-14T04:53:15.350ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1374, tv_nsec: 124692975 } })
85582024-11-14T04:53:15.350ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
85592024-11-14T04:53:15.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85602024-11-14T04:53:15.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
85612024-11-14T04:53:15.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85622024-11-14T04:53:15.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85632024-11-14T04:53:15.400ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85642024-11-14T04:53:15.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
85652024-11-14T04:53:15.405ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85662024-11-14T04:53:15.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85672024-11-14T04:53:15.407ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
85682024-11-14T04:53:15.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85692024-11-14T04:53:15.408ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85702024-11-14T04:53:15.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85712024-11-14T04:53:15.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85722024-11-14T04:53:15.410ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85732024-11-14T04:53:15.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
85742024-11-14T04:53:15.411ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
85752024-11-14T04:53:15.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
85762024-11-14T04:53:15.412ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
85772024-11-14T04:53:15.412ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
85782024-11-14T04:53:15.412ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
85792024-11-14T04:53:15.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
85802024-11-14T04:53:15.412ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85812024-11-14T04:53:15.413ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
85822024-11-14T04:53:15.413ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
85832024-11-14T04:53:15.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
85842024-11-14T04:53:15.414ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85852024-11-14T04:53:15.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
85862024-11-14T04:53:15.417ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85872024-11-14T04:53:15.417ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
85882024-11-14T04:53:15.417ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
85892024-11-14T04:53:15.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85902024-11-14T04:53:15.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
85912024-11-14T04:53:15.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85922024-11-14T04:53:15.419ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
85932024-11-14T04:53:15.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
85942024-11-14T04:53:15.422ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
85952024-11-14T04:53:15.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
85962024-11-14T04:53:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
85972024-11-14T04:53:15.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85982024-11-14T04:53:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85992024-11-14T04:53:15.425ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
86002024-11-14T04:53:15.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
86012024-11-14T04:53:15.427ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86022024-11-14T04:53:15.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86032024-11-14T04:53:15.428ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
86042024-11-14T04:53:15.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
86052024-11-14T04:53:15.429ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86062024-11-14T04:53:15.429ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86072024-11-14T04:53:15.429ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 111.551471ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
86082024-11-14T04:53:15.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
86092024-11-14T04:53:15.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
86102024-11-14T04:53:15.447ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
86112024-11-14T04:53:15.447ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
86122024-11-14T04:53:15.447ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 129.444888ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
86132024-11-14T04:53:15.449ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86142024-11-14T04:53:15.449ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
86152024-11-14T04:53:15.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
86162024-11-14T04:53:15.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
86172024-11-14T04:53:15.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
86182024-11-14T04:53:15.454ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 135.956917ms iteration = 15 status = {"errors":[],"records_set_to_done":[]}
86192024-11-14T04:53:15.457ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86202024-11-14T04:53:15.461ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
86212024-11-14T04:53:15.461ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86222024-11-14T04:53:15.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86232024-11-14T04:53:15.475ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
86242024-11-14T04:53:15.477ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 159.87984ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
86252024-11-14T04:53:15.477ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
86262024-11-14T04:53:15.477ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 160.372654ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
86272024-11-14T04:53:15.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
86282024-11-14T04:53:15.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
86292024-11-14T04:53:15.486ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
86302024-11-14T04:53:15.486ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
86312024-11-14T04:53:15.486ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 169.220901ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
86322024-11-14T04:53:15.488ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
86332024-11-14T04:53:15.488ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
86342024-11-14T04:53:15.488ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
86352024-11-14T04:53:15.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
86362024-11-14T04:53:15.493ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86372024-11-14T04:53:15.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86382024-11-14T04:53:15.496ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86392024-11-14T04:53:15.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86402024-11-14T04:53:15.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 190.800636ms iteration = 15 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
86412024-11-14T04:53:15.512ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 195.364738ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
86422024-11-14T04:53:15.514ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
86432024-11-14T04:53:15.514ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
86442024-11-14T04:53:15.514ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
86452024-11-14T04:53:15.514ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
86462024-11-14T04:53:15.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
86472024-11-14T04:53:15.517ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
86482024-11-14T04:53:15.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
86492024-11-14T04:53:15.520ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
86502024-11-14T04:53:15.520ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
86512024-11-14T04:53:15.520ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e30ddf5-021f-4040-8d8e-4a56c8cf6cb6", "content-length": "26", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
86522024-11-14T04:53:15.520ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
86532024-11-14T04:53:15.523ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
86542024-11-14T04:53:15.525ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
86552024-11-14T04:53:15.525ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
86562024-11-14T04:53:15.525ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
86572024-11-14T04:53:15.525ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
86582024-11-14T04:53:15.525ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
86592024-11-14T04:53:15.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
86602024-11-14T04:53:15.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
86612024-11-14T04:53:15.527ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86622024-11-14T04:53:15.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
86632024-11-14T04:53:15.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
86642024-11-14T04:53:15.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
86652024-11-14T04:53:15.532ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
86662024-11-14T04:53:15.532ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 214.404108ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
86672024-11-14T04:53:15.536ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86682024-11-14T04:53:15.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
86692024-11-14T04:53:15.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
86702024-11-14T04:53:15.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
86712024-11-14T04:53:15.554ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
86722024-11-14T04:53:15.554ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
86732024-11-14T04:53:15.554ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 237.462407ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
86742024-11-14T04:53:15.556ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
86752024-11-14T04:53:15.557ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86762024-11-14T04:53:15.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
86772024-11-14T04:53:15.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86782024-11-14T04:53:15.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
86792024-11-14T04:53:15.561ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86802024-11-14T04:53:15.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86812024-11-14T04:53:15.563ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
86822024-11-14T04:53:15.563ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
86832024-11-14T04:53:15.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
86842024-11-14T04:53:15.564ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
86852024-11-14T04:53:15.564ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
86862024-11-14T04:53:15.564ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f49fab21-41f5-4222-b998-3626b014a152", "content-length": "26", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
86872024-11-14T04:53:15.564ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
86882024-11-14T04:53:15.564ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 247.357154ms iteration = 15 status = 0
86892024-11-14T04:53:15.566ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
86902024-11-14T04:53:15.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 256.159678ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
86912024-11-14T04:53:15.590ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
86922024-11-14T04:53:15.590ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
86932024-11-14T04:53:15.590ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
86942024-11-14T04:53:15.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
86952024-11-14T04:53:15.592ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86962024-11-14T04:53:15.592ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
86972024-11-14T04:53:15.592ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 275.21667ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
86982024-11-14T04:53:15.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
86992024-11-14T04:53:15.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
87002024-11-14T04:53:15.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
87012024-11-14T04:53:15.610ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 293.273963ms iteration = 17 status = {"physical_disks_added":0}
87022024-11-14T04:53:15.617ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
87032024-11-14T04:53:15.617ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
87042024-11-14T04:53:15.617ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
87052024-11-14T04:53:15.617ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 299.937686ms iteration = 9 status = {"generation":1}
87062024-11-14T04:53:15.619ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
87072024-11-14T04:53:15.619ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
87082024-11-14T04:53:15.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
87092024-11-14T04:53:15.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
87102024-11-14T04:53:15.633ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
87112024-11-14T04:53:15.633ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 315.927919ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
87122024-11-14T04:53:15.641ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
87132024-11-14T04:53:15.688ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
87142024-11-14T04:53:15.688ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c1d28ff4-4286-48c2-94d7-71086aa09c7f", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87152024-11-14T04:53:15.689ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87162024-11-14T04:53:15.691ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3760eea0-2033-4cac-a55d-bbc9e30b48b3", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87172024-11-14T04:53:15.691ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87182024-11-14T04:53:15.693ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87192024-11-14T04:53:15.693ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 376.310047ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
87202024-11-14T04:53:15.706ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
87212024-11-14T04:53:15.706ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
87222024-11-14T04:53:15.706ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1280f1e-8db9-4ea4-ab7c-665a986a5256", "content-length": "26", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
87232024-11-14T04:53:15.706ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
87242024-11-14T04:53:15.708ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
87252024-11-14T04:53:15.717ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87262024-11-14T04:53:15.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87272024-11-14T04:53:15.735ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
87282024-11-14T04:53:15.735ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
87292024-11-14T04:53:15.735ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ff0c5c7-9ba0-4b7d-b1e2-7ad24cad2f8d", "content-length": "372", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
87302024-11-14T04:53:15.735ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87312024-11-14T04:53:15.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87322024-11-14T04:53:15.737ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87332024-11-14T04:53:15.737ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87342024-11-14T04:53:15.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
87352024-11-14T04:53:15.740ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87362024-11-14T04:53:15.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87372024-11-14T04:53:15.740ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
87382024-11-14T04:53:15.740ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
87392024-11-14T04:53:15.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
87402024-11-14T04:53:15.742ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
87412024-11-14T04:53:15.742ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
87422024-11-14T04:53:15.742ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
87432024-11-14T04:53:15.742ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 425.576184ms iteration = 9 status = {"generation":2}
87442024-11-14T04:53:15.768ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87452024-11-14T04:53:15.770ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
87462024-11-14T04:53:15.770ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87472024-11-14T04:53:15.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87482024-11-14T04:53:15.780ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
87492024-11-14T04:53:15.780ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
87502024-11-14T04:53:15.780ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2db00bbb-555d-4c27-bb9f-b6d9c9a38c8b", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
87512024-11-14T04:53:15.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87522024-11-14T04:53:15.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87532024-11-14T04:53:15.781ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
87542024-11-14T04:53:15.782ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5c4f0a23-6636-4346-b754-c3c1c355aa29", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
87552024-11-14T04:53:15.782ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
87562024-11-14T04:53:15.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87572024-11-14T04:53:15.783ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "a2b12aae-0c19-4cad-bb26-0feeaf66e13d", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
87582024-11-14T04:53:15.783ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
87592024-11-14T04:53:15.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87602024-11-14T04:53:15.785ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7886572c-d59c-4a4e-822c-9d00983d53cf", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
87612024-11-14T04:53:15.785ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
87622024-11-14T04:53:15.786ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87632024-11-14T04:53:15.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87642024-11-14T04:53:15.787ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
87652024-11-14T04:53:15.787ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 470.541536ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
87662024-11-14T04:53:15.790ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
87672024-11-14T04:53:15.790ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
87682024-11-14T04:53:15.790ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1f097234-6faf-42d0-902a-c3238e1d0046", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
87692024-11-14T04:53:15.791ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 474.422399ms iteration = 16 status = {}
87702024-11-14T04:53:15.802ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
87712024-11-14T04:53:15.802ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb29e0be-30f9-46b2-b883-feee9c13e4a5", "content-length": "1", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
87722024-11-14T04:53:15.802ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 485.342794ms iteration = 15 status = {}
87732024-11-14T04:53:15.803ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
87742024-11-14T04:53:15.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
87752024-11-14T04:53:15.806ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 488.829345ms iteration = 15 status = null
87762024-11-14T04:53:15.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87772024-11-14T04:53:15.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87782024-11-14T04:53:15.875ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
87792024-11-14T04:53:15.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87802024-11-14T04:53:15.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87812024-11-14T04:53:15.878ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
87822024-11-14T04:53:15.878ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c1f771a-dda2-4a9e-a2e0-422c93dfa40b", "content-length": "691", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
87832024-11-14T04:53:15.878ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
87842024-11-14T04:53:15.878ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
87852024-11-14T04:53:15.878ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
87862024-11-14T04:53:15.879ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 561.775617ms iteration = 15 status = {}
87872024-11-14T04:53:15.886ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
87882024-11-14T04:53:15.886ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe002762-1393-4ad4-a8ae-63cd281d6bf7", "content-length": "134", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
87892024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
87902024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
87912024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
87922024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
87932024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
87942024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
87952024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
87962024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
87972024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
87982024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
87992024-11-14T04:53:15.886ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
88002024-11-14T04:53:15.886ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88012024-11-14T04:53:15.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88022024-11-14T04:53:15.889ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
88032024-11-14T04:53:15.889ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "76df3c96-629f-444e-b871-2a80deaaafbc", "content-length": "2", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
88042024-11-14T04:53:15.889ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
88052024-11-14T04:53:15.889ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "602c06be-96dd-4ce5-a7ef-3f2186d44835", "content-length": "51", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
88062024-11-14T04:53:15.889ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
88072024-11-14T04:53:15.889ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
88082024-11-14T04:53:16.070ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "33244eb4-8849-47a2-b87a-c8cd21f596c1", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
88092024-11-14T04:53:16.071ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
88102024-11-14T04:53:16.071ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
88112024-11-14T04:53:16.074ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "be990ebe-dd7d-47dd-8ba9-f4cc9e51128a", "content-length": "682", "date": "Thu, 14 Nov 2024 04:53:15 GMT"} })
88122024-11-14T04:53:16.074ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88132024-11-14T04:53:16.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88142024-11-14T04:53:16.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88152024-11-14T04:53:16.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88162024-11-14T04:53:16.081ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88172024-11-14T04:53:16.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88182024-11-14T04:53:16.083ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88192024-11-14T04:53:16.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88202024-11-14T04:53:16.090ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 772.711596ms iteration = 15 status = {}
88212024-11-14T04:53:20.259ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1130 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 3e823c49-91fc-43b5-a517-fcfd57ebc69e response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
88222024-11-14T04:53:25.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
88232024-11-14T04:53:25.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
88242024-11-14T04:53:25.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88252024-11-14T04:53:25.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88262024-11-14T04:53:25.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88272024-11-14T04:53:25.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88282024-11-14T04:53:25.347ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88292024-11-14T04:53:25.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88302024-11-14T04:53:25.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88312024-11-14T04:53:25.349ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
88322024-11-14T04:53:25.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88332024-11-14T04:53:25.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88342024-11-14T04:53:25.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88352024-11-14T04:53:25.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88362024-11-14T04:53:25.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88372024-11-14T04:53:25.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
88382024-11-14T04:53:25.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88392024-11-14T04:53:25.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88402024-11-14T04:53:25.371ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
88412024-11-14T04:53:25.371ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 53.747666ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
88422024-11-14T04:53:25.382ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.624942ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
88432024-11-14T04:53:35.259ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1540 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 4af3a00f-0dd3-44e5-837d-cf110e7e2a50 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
88442024-11-14T04:53:35.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
88452024-11-14T04:53:35.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
88462024-11-14T04:53:35.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88472024-11-14T04:53:35.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88482024-11-14T04:53:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88492024-11-14T04:53:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88502024-11-14T04:53:35.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88512024-11-14T04:53:35.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88522024-11-14T04:53:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88532024-11-14T04:53:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
88542024-11-14T04:53:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88552024-11-14T04:53:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88562024-11-14T04:53:35.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.951273ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
88572024-11-14T04:53:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88582024-11-14T04:53:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88592024-11-14T04:53:35.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88602024-11-14T04:53:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
88612024-11-14T04:53:35.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88622024-11-14T04:53:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88632024-11-14T04:53:35.375ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
88642024-11-14T04:53:35.375ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.804634ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
88652024-11-14T04:53:40.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 7cdd4eb6-a49c-40cf-9527-103eb852dca0 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
88662024-11-14T04:53:40.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 7cdd4eb6-a49c-40cf-9527-103eb852dca0 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
88672024-11-14T04:53:40.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3115 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 7cdd4eb6-a49c-40cf-9527-103eb852dca0 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
88682024-11-14T04:53:40.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 6f7f3e5d-59b2-46b9-af65-c809cbeb6dd8 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
88692024-11-14T04:53:40.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 6f7f3e5d-59b2-46b9-af65-c809cbeb6dd8 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
88702024-11-14T04:53:40.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2189 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 6f7f3e5d-59b2-46b9-af65-c809cbeb6dd8 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
88712024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
88722024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
88732024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
88742024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
88752024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88762024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
88772024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
88782024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
88792024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
88802024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
88812024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
88822024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88832024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
88842024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88852024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88862024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
88872024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
88882024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
88892024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
88902024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
88912024-11-14T04:53:45.317ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
88922024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88932024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
88942024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
88952024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
88962024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
88972024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
88982024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88992024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
89002024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89012024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
89022024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
89032024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
89042024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89052024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
89062024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89072024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
89082024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
89092024-11-14T04:53:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89102024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
89112024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
89122024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
89132024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
89142024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
89152024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
89162024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
89172024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
89182024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
89192024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
89202024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
89212024-11-14T04:53:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
89222024-11-14T04:53:45.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89232024-11-14T04:53:45.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89242024-11-14T04:53:45.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
89252024-11-14T04:53:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
89262024-11-14T04:53:45.321ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.261313ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
89272024-11-14T04:53:45.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
89282024-11-14T04:53:45.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89292024-11-14T04:53:45.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89302024-11-14T04:53:45.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
89312024-11-14T04:53:45.323ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89322024-11-14T04:53:45.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89332024-11-14T04:53:45.326ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89342024-11-14T04:53:45.326ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1404, tv_nsec: 101521370 } })
89352024-11-14T04:53:45.326ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89362024-11-14T04:53:45.326ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
89372024-11-14T04:53:45.327ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1404, tv_nsec: 102694924 } })
89382024-11-14T04:53:45.328ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
89392024-11-14T04:53:45.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89402024-11-14T04:53:45.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
89412024-11-14T04:53:45.328ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89422024-11-14T04:53:45.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
89432024-11-14T04:53:45.328ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
89442024-11-14T04:53:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89452024-11-14T04:53:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
89462024-11-14T04:53:45.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
89472024-11-14T04:53:45.330ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89482024-11-14T04:53:45.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89492024-11-14T04:53:45.333ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 15.833588ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
89502024-11-14T04:53:45.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89512024-11-14T04:53:45.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89522024-11-14T04:53:45.339ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1404, tv_nsec: 113980091 } })
89532024-11-14T04:53:45.339ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
89542024-11-14T04:53:45.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89552024-11-14T04:53:45.344ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89562024-11-14T04:53:45.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89572024-11-14T04:53:45.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
89582024-11-14T04:53:45.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89592024-11-14T04:53:45.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89602024-11-14T04:53:45.384ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
89612024-11-14T04:53:45.384ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 66.99494ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
89622024-11-14T04:53:45.384ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89632024-11-14T04:53:45.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
89642024-11-14T04:53:45.388ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89652024-11-14T04:53:45.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89662024-11-14T04:53:45.391ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89672024-11-14T04:53:45.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89682024-11-14T04:53:45.400ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
89692024-11-14T04:53:45.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
89702024-11-14T04:53:45.401ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
89712024-11-14T04:53:45.401ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
89722024-11-14T04:53:45.401ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
89732024-11-14T04:53:45.401ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
89742024-11-14T04:53:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
89752024-11-14T04:53:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
89762024-11-14T04:53:45.402ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 84.509107ms iteration = 16 status = {"errors":[],"records_set_to_done":[]}
89772024-11-14T04:53:45.408ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89782024-11-14T04:53:45.411ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
89792024-11-14T04:53:45.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89802024-11-14T04:53:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89812024-11-14T04:53:45.415ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
89822024-11-14T04:53:45.415ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 97.537435ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
89832024-11-14T04:53:45.423ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 104.784834ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
89842024-11-14T04:53:45.431ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89852024-11-14T04:53:45.432ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
89862024-11-14T04:53:45.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
89872024-11-14T04:53:45.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
89882024-11-14T04:53:45.433ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89892024-11-14T04:53:45.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89902024-11-14T04:53:45.441ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
89912024-11-14T04:53:45.441ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
89922024-11-14T04:53:45.442ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cf6ff2c-f69b-4d86-b303-21b6223607f2", "content-length": "26", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
89932024-11-14T04:53:45.442ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
89942024-11-14T04:53:45.443ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
89952024-11-14T04:53:45.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
89962024-11-14T04:53:45.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
89972024-11-14T04:53:45.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89982024-11-14T04:53:45.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89992024-11-14T04:53:45.451ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
90002024-11-14T04:53:45.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
90012024-11-14T04:53:45.454ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90022024-11-14T04:53:45.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
90032024-11-14T04:53:45.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
90042024-11-14T04:53:45.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
90052024-11-14T04:53:45.460ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 142.926629ms iteration = 16 status = 0
90062024-11-14T04:53:45.465ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 147.513522ms iteration = 16 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
90072024-11-14T04:53:45.466ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
90082024-11-14T04:53:45.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
90092024-11-14T04:53:45.467ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
90102024-11-14T04:53:45.467ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
90112024-11-14T04:53:45.468ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7f4e82f-2ea0-444e-9484-9697f1e2a68c", "content-length": "26", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
90122024-11-14T04:53:45.468ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
90132024-11-14T04:53:45.468ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
90142024-11-14T04:53:45.468ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
90152024-11-14T04:53:45.468ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c66917ec-ef66-4196-a4ec-ae994a9de607", "content-length": "26", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
90162024-11-14T04:53:45.468ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
90172024-11-14T04:53:45.469ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90182024-11-14T04:53:45.469ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
90192024-11-14T04:53:45.469ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
90202024-11-14T04:53:45.469ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
90212024-11-14T04:53:45.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
90222024-11-14T04:53:45.470ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
90232024-11-14T04:53:45.501ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 184.297807ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
90242024-11-14T04:53:45.508ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 190.246209ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
90252024-11-14T04:53:45.512ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90262024-11-14T04:53:45.512ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
90272024-11-14T04:53:45.512ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 195.78704ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
90282024-11-14T04:53:45.530ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
90292024-11-14T04:53:45.530ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "183d8aa0-7853-4e89-a520-5d2f607b0c46", "content-length": "1", "date": "Thu, 14 Nov 2024 04:53:44 GMT"} })
90302024-11-14T04:53:45.533ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
90312024-11-14T04:53:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
90322024-11-14T04:53:45.538ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 221.00702ms iteration = 16 status = null
90332024-11-14T04:53:45.541ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 223.777771ms iteration = 16 status = {}
90342024-11-14T04:53:45.559ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 242.591875ms iteration = 18 status = {"physical_disks_added":0}
90352024-11-14T04:53:45.572ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90362024-11-14T04:53:45.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
90372024-11-14T04:53:45.588ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
90382024-11-14T04:53:45.588ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
90392024-11-14T04:53:45.588ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "607d6768-72ad-487f-99ad-83a6156fdf73", "content-length": "372", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
90402024-11-14T04:53:45.588ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90412024-11-14T04:53:45.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90422024-11-14T04:53:45.589ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90432024-11-14T04:53:45.591ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
90442024-11-14T04:53:45.591ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90452024-11-14T04:53:45.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
90462024-11-14T04:53:45.593ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90472024-11-14T04:53:45.596ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
90482024-11-14T04:53:45.596ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90492024-11-14T04:53:45.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
90502024-11-14T04:53:45.621ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90512024-11-14T04:53:45.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90522024-11-14T04:53:45.623ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
90532024-11-14T04:53:45.623ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90542024-11-14T04:53:45.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90552024-11-14T04:53:45.629ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
90562024-11-14T04:53:45.629ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
90572024-11-14T04:53:45.629ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2e1bc087-4352-4817-a1da-9e340e373da8", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
90582024-11-14T04:53:45.629ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90592024-11-14T04:53:45.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90602024-11-14T04:53:45.632ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90612024-11-14T04:53:45.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90622024-11-14T04:53:45.633ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
90632024-11-14T04:53:45.633ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "8118a91f-37f9-4410-b60e-a4fd011aeb05", "content-length": "134", "date": "Thu, 14 Nov 2024 04:53:44 GMT"} })
90642024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
90652024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
90662024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
90672024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
90682024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90692024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90702024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90712024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
90722024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90732024-11-14T04:53:45.633ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
90742024-11-14T04:53:45.634ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90752024-11-14T04:53:45.634ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
90762024-11-14T04:53:45.634ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90772024-11-14T04:53:45.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90782024-11-14T04:53:45.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90792024-11-14T04:53:45.641ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
90802024-11-14T04:53:45.642ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2726d04a-aa90-4d9c-8158-5e23b58943fc", "content-length": "691", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
90812024-11-14T04:53:45.642ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
90822024-11-14T04:53:45.642ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
90832024-11-14T04:53:45.642ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
90842024-11-14T04:53:45.644ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
90852024-11-14T04:53:45.644ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "8848f202-0a1c-4e83-9dac-5f516cd439c3", "content-length": "2", "date": "Thu, 14 Nov 2024 04:53:44 GMT"} })
90862024-11-14T04:53:45.644ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
90872024-11-14T04:53:45.644ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 327.430482ms iteration = 16 status = {}
90882024-11-14T04:53:45.644ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac4f76f2-0393-4d0d-90db-fce8e3952a75", "content-length": "51", "date": "Thu, 14 Nov 2024 04:53:44 GMT"} })
90892024-11-14T04:53:45.644ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
90902024-11-14T04:53:45.644ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
90912024-11-14T04:53:45.645ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
90922024-11-14T04:53:45.645ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
90932024-11-14T04:53:45.764ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2c71d5bc-71c9-4e60-a44f-3262e269ac99", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
90942024-11-14T04:53:45.764ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
90952024-11-14T04:53:45.764ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
90962024-11-14T04:53:45.765ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "addb2410-4d12-49bf-a401-1db6f470c5c1", "content-length": "682", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
90972024-11-14T04:53:45.765ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90982024-11-14T04:53:45.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90992024-11-14T04:53:45.769ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91002024-11-14T04:53:45.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91012024-11-14T04:53:45.770ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "29e877c1-a8e9-45eb-8a3e-fc0660b8c637", "date": "Thu, 14 Nov 2024 04:53:45 GMT"} })
91022024-11-14T04:53:45.772ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91032024-11-14T04:53:45.772ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 455.331996ms iteration = 17 status = {}
91042024-11-14T04:53:45.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91052024-11-14T04:53:45.774ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91062024-11-14T04:53:45.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91072024-11-14T04:53:45.781ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 463.77889ms iteration = 16 status = {}
91082024-11-14T04:53:50.155ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
91092024-11-14T04:53:50.155ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
91102024-11-14T04:53:50.191ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
91112024-11-14T04:53:50.191ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:36485
91122024-11-14T04:53:50.191ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:36485 req_id = f9c7a087-c84f-46f7-b8d4-6735f141cf3b roles = RoleSet { roles: {} } uri = /metrics/producers
91132024-11-14T04:53:50.191ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:36485 req_id = f9c7a087-c84f-46f7-b8d4-6735f141cf3b resource = Database result = Ok(()) uri = /metrics/producers
91142024-11-14T04:53:50.240ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
91152024-11-14T04:53:50.240ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49234 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:36485 req_id = f9c7a087-c84f-46f7-b8d4-6735f141cf3b response_code = 201 uri = /metrics/producers
91162024-11-14T04:53:50.240ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "f9c7a087-c84f-46f7-b8d4-6735f141cf3b", "content-length": "41", "date": "Thu, 14 Nov 2024 04:53:50 GMT"} })
91172024-11-14T04:53:50.242ZDEBGnexus (producer-registration-task): registered with nexus successfully
91182024-11-14T04:53:50.242ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
91192024-11-14T04:53:50.259ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1158 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 2e4d18ea-99ce-4e71-93bf-500dbd200f1d response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
91202024-11-14T04:53:55.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
91212024-11-14T04:53:55.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
91222024-11-14T04:53:55.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91232024-11-14T04:53:55.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91242024-11-14T04:53:55.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91252024-11-14T04:53:55.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91262024-11-14T04:53:55.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91272024-11-14T04:53:55.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
91282024-11-14T04:53:55.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91292024-11-14T04:53:55.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91302024-11-14T04:53:55.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91312024-11-14T04:53:55.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91322024-11-14T04:53:55.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91332024-11-14T04:53:55.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91342024-11-14T04:53:55.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91352024-11-14T04:53:55.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
91362024-11-14T04:53:55.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91372024-11-14T04:53:55.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91382024-11-14T04:53:55.356ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.158984ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
91392024-11-14T04:53:55.368ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
91402024-11-14T04:53:55.368ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.75321ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
91412024-11-14T04:54:05.259ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1344 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 4e1c4311-618f-4404-91a9-31dc6f181f88 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
91422024-11-14T04:54:05.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
91432024-11-14T04:54:05.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91442024-11-14T04:54:05.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
91452024-11-14T04:54:05.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91462024-11-14T04:54:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91472024-11-14T04:54:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91482024-11-14T04:54:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91492024-11-14T04:54:05.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91502024-11-14T04:54:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91512024-11-14T04:54:05.321ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.275294ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
91522024-11-14T04:54:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
91532024-11-14T04:54:05.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91542024-11-14T04:54:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91552024-11-14T04:54:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91562024-11-14T04:54:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91572024-11-14T04:54:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91582024-11-14T04:54:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
91592024-11-14T04:54:05.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91602024-11-14T04:54:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91612024-11-14T04:54:05.335ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
91622024-11-14T04:54:05.335ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.156926ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
91632024-11-14T04:54:10.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = a19a5744-8363-4325-81dc-5fff3c6b4748 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
91642024-11-14T04:54:10.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = a19a5744-8363-4325-81dc-5fff3c6b4748 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
91652024-11-14T04:54:10.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2380 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = a19a5744-8363-4325-81dc-5fff3c6b4748 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
91662024-11-14T04:54:10.253ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = b875720c-55dc-45d9-a9c2-94277dc33323 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
91672024-11-14T04:54:10.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = b875720c-55dc-45d9-a9c2-94277dc33323 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
91682024-11-14T04:54:10.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1742 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = b875720c-55dc-45d9-a9c2-94277dc33323 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
91692024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
91702024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
91712024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91722024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
91732024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
91742024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
91752024-11-14T04:54:15.317ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:44:15.317486167 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
91762024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
91772024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
91782024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
91792024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
91802024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
91812024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
91822024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
91832024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
91842024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
91852024-11-14T04:54:15.317ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
91862024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91872024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
91882024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
91892024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
91902024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91912024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
91922024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
91932024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
91942024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
91952024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
91962024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
91972024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
91982024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
91992024-11-14T04:54:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
92002024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
92012024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
92022024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92032024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
92042024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
92052024-11-14T04:54:15.318ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
92062024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
92072024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92082024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
92092024-11-14T04:54:15.318ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
92102024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
92112024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
92122024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.96\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
92132024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
92142024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
92152024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
92162024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92172024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
92182024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
92192024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
92202024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
92212024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
92222024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92232024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
92242024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
92252024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
92262024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92272024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
92282024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
92292024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
92302024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
92312024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
92322024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
92332024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
92342024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
92352024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
92362024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
92372024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
92382024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
92392024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
92402024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
92412024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
92422024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
92432024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
92442024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92452024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
92462024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92472024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
92482024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92492024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
92502024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
92512024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
92522024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
92532024-11-14T04:54:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
92542024-11-14T04:54:15.318ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1434, tv_nsec: 93963489 } })
92552024-11-14T04:54:15.318ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1434, tv_nsec: 94005850 } })
92562024-11-14T04:54:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
92572024-11-14T04:54:15.319ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
92582024-11-14T04:54:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.759941ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
92592024-11-14T04:54:15.319ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
92602024-11-14T04:54:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.871694ms iteration = 10 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
92612024-11-14T04:54:15.320ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
92622024-11-14T04:54:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
92632024-11-14T04:54:15.321ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:44:15.317486167 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
92642024-11-14T04:54:15.321ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.178841ms iteration = 9 status = {"expiration":"2024-11-14T04:44:15.317486167Z","pruned":[]}
92652024-11-14T04:54:15.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92662024-11-14T04:54:15.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92672024-11-14T04:54:15.322ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.235832ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
92682024-11-14T04:54:15.323ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1434, tv_nsec: 98578282 } })
92692024-11-14T04:54:15.323ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
92702024-11-14T04:54:15.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
92712024-11-14T04:54:15.324ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
92722024-11-14T04:54:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92732024-11-14T04:54:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
92742024-11-14T04:54:15.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92752024-11-14T04:54:15.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92762024-11-14T04:54:15.328ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92772024-11-14T04:54:15.329ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1434, tv_nsec: 105081151 } })
92782024-11-14T04:54:15.330ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
92792024-11-14T04:54:15.330ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92802024-11-14T04:54:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
92812024-11-14T04:54:15.330ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
92822024-11-14T04:54:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
92832024-11-14T04:54:15.333ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
92842024-11-14T04:54:15.333ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
92852024-11-14T04:54:15.333ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92862024-11-14T04:54:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
92872024-11-14T04:54:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
92882024-11-14T04:54:15.335ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92892024-11-14T04:54:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92902024-11-14T04:54:15.340ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.950256ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
92912024-11-14T04:54:15.341ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92922024-11-14T04:54:15.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
92932024-11-14T04:54:15.343ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92942024-11-14T04:54:15.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92952024-11-14T04:54:15.344ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92962024-11-14T04:54:15.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92972024-11-14T04:54:15.361ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 43.512591ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
92982024-11-14T04:54:15.366ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92992024-11-14T04:54:15.367ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93002024-11-14T04:54:15.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93012024-11-14T04:54:15.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93022024-11-14T04:54:15.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
93032024-11-14T04:54:15.370ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
93042024-11-14T04:54:15.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
93052024-11-14T04:54:15.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
93062024-11-14T04:54:15.376ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93072024-11-14T04:54:15.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
93082024-11-14T04:54:15.380ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93092024-11-14T04:54:15.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93102024-11-14T04:54:15.385ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93112024-11-14T04:54:15.385ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1434, tv_nsec: 160407373 } })
93122024-11-14T04:54:15.385ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
93132024-11-14T04:54:15.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
93142024-11-14T04:54:15.403ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93152024-11-14T04:54:15.406ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93162024-11-14T04:54:15.407ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
93172024-11-14T04:54:15.407ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93182024-11-14T04:54:15.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93192024-11-14T04:54:15.409ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93202024-11-14T04:54:15.414ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
93212024-11-14T04:54:15.414ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93222024-11-14T04:54:15.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93232024-11-14T04:54:15.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
93242024-11-14T04:54:15.419ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
93252024-11-14T04:54:15.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
93262024-11-14T04:54:15.421ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93272024-11-14T04:54:15.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
93282024-11-14T04:54:15.422ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93292024-11-14T04:54:15.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93302024-11-14T04:54:15.436ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
93312024-11-14T04:54:15.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
93322024-11-14T04:54:15.439ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
93332024-11-14T04:54:15.439ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
93342024-11-14T04:54:15.439ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
93352024-11-14T04:54:15.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
93362024-11-14T04:54:15.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
93372024-11-14T04:54:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
93382024-11-14T04:54:15.442ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 124.752243ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
93392024-11-14T04:54:15.443ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 125.35768ms iteration = 17 status = {"errors":[],"records_set_to_done":[]}
93402024-11-14T04:54:15.460ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
93412024-11-14T04:54:15.460ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
93422024-11-14T04:54:15.460ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
93432024-11-14T04:54:15.460ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
93442024-11-14T04:54:15.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
93452024-11-14T04:54:15.463ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
93462024-11-14T04:54:15.463ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 145.518474ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
93472024-11-14T04:54:15.464ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
93482024-11-14T04:54:15.464ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
93492024-11-14T04:54:15.464ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 146.469752ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
93502024-11-14T04:54:15.465ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
93512024-11-14T04:54:15.465ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
93522024-11-14T04:54:15.466ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "60019505-3918-49d9-ae06-c640623a6b7f", "content-length": "26", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
93532024-11-14T04:54:15.466ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
93542024-11-14T04:54:15.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93552024-11-14T04:54:15.468ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
93562024-11-14T04:54:15.471ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
93572024-11-14T04:54:15.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
93582024-11-14T04:54:15.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93592024-11-14T04:54:15.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93602024-11-14T04:54:15.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
93612024-11-14T04:54:15.474ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93622024-11-14T04:54:15.476ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 158.310404ms iteration = 19 status = {"physical_disks_added":0}
93632024-11-14T04:54:15.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
93642024-11-14T04:54:15.479ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93652024-11-14T04:54:15.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
93662024-11-14T04:54:15.479ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
93672024-11-14T04:54:15.479ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
93682024-11-14T04:54:15.479ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
93692024-11-14T04:54:15.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
93702024-11-14T04:54:15.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
93712024-11-14T04:54:15.487ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
93722024-11-14T04:54:15.488ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
93732024-11-14T04:54:15.488ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
93742024-11-14T04:54:15.488ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
93752024-11-14T04:54:15.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
93762024-11-14T04:54:15.490ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
93772024-11-14T04:54:15.490ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
93782024-11-14T04:54:15.490ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a55a183-6ee0-44d8-8314-b3083ad49fdc", "content-length": "26", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
93792024-11-14T04:54:15.490ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
93802024-11-14T04:54:15.493ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
93812024-11-14T04:54:15.493ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
93822024-11-14T04:54:15.494ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
93832024-11-14T04:54:15.494ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 175.968956ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
93842024-11-14T04:54:15.495ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 177.15275ms iteration = 17 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
93852024-11-14T04:54:15.513ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93862024-11-14T04:54:15.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93872024-11-14T04:54:15.517ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
93882024-11-14T04:54:15.517ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 199.624251ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
93892024-11-14T04:54:15.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93902024-11-14T04:54:15.518ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
93912024-11-14T04:54:15.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93922024-11-14T04:54:15.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
93932024-11-14T04:54:15.528ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
93942024-11-14T04:54:15.528ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
93952024-11-14T04:54:15.528ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
93962024-11-14T04:54:15.528ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 211.501494ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
93972024-11-14T04:54:15.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
93982024-11-14T04:54:15.544ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93992024-11-14T04:54:15.545ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94002024-11-14T04:54:15.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
94012024-11-14T04:54:15.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
94022024-11-14T04:54:15.549ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
94032024-11-14T04:54:15.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
94042024-11-14T04:54:15.554ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94052024-11-14T04:54:15.556ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94062024-11-14T04:54:15.557ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
94072024-11-14T04:54:15.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
94082024-11-14T04:54:15.558ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
94092024-11-14T04:54:15.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
94102024-11-14T04:54:15.559ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
94112024-11-14T04:54:15.559ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
94122024-11-14T04:54:15.559ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94132024-11-14T04:54:15.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
94142024-11-14T04:54:15.560ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
94152024-11-14T04:54:15.560ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
94162024-11-14T04:54:15.560ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e88a0f5d-4029-4d82-9241-4d98adeedcd2", "content-length": "26", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
94172024-11-14T04:54:15.560ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
94182024-11-14T04:54:15.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
94192024-11-14T04:54:15.562ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
94202024-11-14T04:54:15.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
94212024-11-14T04:54:15.563ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
94222024-11-14T04:54:15.564ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 246.792087ms iteration = 17 status = 0
94232024-11-14T04:54:15.567ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
94242024-11-14T04:54:15.567ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
94252024-11-14T04:54:15.567ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 249.661439ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
94262024-11-14T04:54:15.568ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 249.847836ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
94272024-11-14T04:54:15.590ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
94282024-11-14T04:54:15.590ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 271.843273ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
94292024-11-14T04:54:15.615ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
94302024-11-14T04:54:15.615ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
94312024-11-14T04:54:15.615ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
94322024-11-14T04:54:15.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
94332024-11-14T04:54:15.617ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
94342024-11-14T04:54:15.617ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
94352024-11-14T04:54:15.617ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
94362024-11-14T04:54:15.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
94372024-11-14T04:54:15.617ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94382024-11-14T04:54:15.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94392024-11-14T04:54:15.620ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 302.094159ms iteration = 17 status = {}
94402024-11-14T04:54:15.622ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
94412024-11-14T04:54:15.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
94422024-11-14T04:54:15.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
94432024-11-14T04:54:15.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
94442024-11-14T04:54:15.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
94452024-11-14T04:54:15.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
94462024-11-14T04:54:15.625ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
94472024-11-14T04:54:15.625ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
94482024-11-14T04:54:15.625ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
94492024-11-14T04:54:15.625ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 307.076263ms iteration = 10 status = {"generation":1}
94502024-11-14T04:54:15.627ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
94512024-11-14T04:54:15.627ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
94522024-11-14T04:54:15.627ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
94532024-11-14T04:54:15.627ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 309.699639ms iteration = 10 status = {"generation":2}
94542024-11-14T04:54:15.628ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
94552024-11-14T04:54:15.628ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 310.673097ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
94562024-11-14T04:54:15.638ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
94572024-11-14T04:54:15.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
94582024-11-14T04:54:15.640ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94592024-11-14T04:54:15.642ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
94602024-11-14T04:54:15.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
94612024-11-14T04:54:15.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
94622024-11-14T04:54:15.652ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94632024-11-14T04:54:15.654ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
94642024-11-14T04:54:15.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
94652024-11-14T04:54:15.654ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
94662024-11-14T04:54:15.654ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ab1dadeb-906e-45c4-90d0-bb35870159cb", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94672024-11-14T04:54:15.654ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94682024-11-14T04:54:15.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
94692024-11-14T04:54:15.656ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "27bdc1ad-afdf-4aeb-91e9-d8c12f799571", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94702024-11-14T04:54:15.656ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94712024-11-14T04:54:15.659ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94722024-11-14T04:54:15.659ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 341.50521ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
94732024-11-14T04:54:15.700ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
94742024-11-14T04:54:15.710ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
94752024-11-14T04:54:15.710ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
94762024-11-14T04:54:15.710ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4db77610-b178-49a5-a543-71ca04dc8957", "content-length": "372", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
94772024-11-14T04:54:15.710ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94782024-11-14T04:54:15.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94792024-11-14T04:54:15.711ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
94802024-11-14T04:54:15.711ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b730eaa0-fa9a-4f71-bf37-f84a3297d5f5", "content-length": "1", "date": "Thu, 14 Nov 2024 04:54:14 GMT"} })
94812024-11-14T04:54:15.713ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
94822024-11-14T04:54:15.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
94832024-11-14T04:54:15.716ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 398.789058ms iteration = 17 status = null
94842024-11-14T04:54:15.725ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
94852024-11-14T04:54:15.725ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
94862024-11-14T04:54:15.725ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "05dac0dd-6b0d-4bdf-86eb-d73806e907ba", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
94872024-11-14T04:54:15.725ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94882024-11-14T04:54:15.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94892024-11-14T04:54:15.727ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94902024-11-14T04:54:15.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94912024-11-14T04:54:15.729ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94922024-11-14T04:54:15.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94932024-11-14T04:54:15.782ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
94942024-11-14T04:54:15.782ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b14a5e17-873f-48f6-bcb9-4107899ea2f4", "content-length": "691", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
94952024-11-14T04:54:15.782ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
94962024-11-14T04:54:15.782ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
94972024-11-14T04:54:15.782ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
94982024-11-14T04:54:15.785ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 467.674533ms iteration = 17 status = {}
94992024-11-14T04:54:15.785ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
95002024-11-14T04:54:15.785ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
95012024-11-14T04:54:15.785ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "82c49129-56a1-4917-89c5-d72b9206c44b", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
95022024-11-14T04:54:15.787ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 470.135094ms iteration = 18 status = {}
95032024-11-14T04:54:15.794ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
95042024-11-14T04:54:15.794ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "18515a3c-3d40-4433-b33a-7d60d6253f73", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
95052024-11-14T04:54:15.794ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
95062024-11-14T04:54:15.796ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a0a0aa03-11b4-4a71-ac7a-31fee2f395fc", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
95072024-11-14T04:54:15.796ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
95082024-11-14T04:54:15.797ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "26effa11-fc68-42ab-aaf5-609e74091201", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
95092024-11-14T04:54:15.797ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
95102024-11-14T04:54:15.800ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
95112024-11-14T04:54:15.800ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 482.208344ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
95122024-11-14T04:54:15.817ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95132024-11-14T04:54:15.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95142024-11-14T04:54:15.819ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
95152024-11-14T04:54:15.819ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95162024-11-14T04:54:15.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95172024-11-14T04:54:15.830ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
95182024-11-14T04:54:15.831ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5091b9e-7f8b-40ff-af0b-bb01b004bf1c", "content-length": "134", "date": "Thu, 14 Nov 2024 04:54:14 GMT"} })
95192024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
95202024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
95212024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
95222024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
95232024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95242024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95252024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95262024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
95272024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95282024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
95292024-11-14T04:54:15.831ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
95302024-11-14T04:54:15.831ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95312024-11-14T04:54:15.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95322024-11-14T04:54:15.833ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
95332024-11-14T04:54:15.833ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e651616-334a-47f1-89b2-cda295bcd420", "content-length": "2", "date": "Thu, 14 Nov 2024 04:54:14 GMT"} })
95342024-11-14T04:54:15.833ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
95352024-11-14T04:54:15.833ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "860884e8-bc04-4607-86af-046e0aa99f79", "content-length": "51", "date": "Thu, 14 Nov 2024 04:54:14 GMT"} })
95362024-11-14T04:54:15.833ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
95372024-11-14T04:54:15.833ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
95382024-11-14T04:54:15.941ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c2e717e8-1eae-4831-8c16-7c5d45493889", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
95392024-11-14T04:54:15.941ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
95402024-11-14T04:54:15.941ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
95412024-11-14T04:54:15.942ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "38159d26-1c99-4334-a735-44b438f07672", "content-length": "682", "date": "Thu, 14 Nov 2024 04:54:15 GMT"} })
95422024-11-14T04:54:15.942ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95432024-11-14T04:54:15.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95442024-11-14T04:54:15.945ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95452024-11-14T04:54:15.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95462024-11-14T04:54:15.949ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95472024-11-14T04:54:15.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95482024-11-14T04:54:15.952ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95492024-11-14T04:54:15.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95502024-11-14T04:54:15.959ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 641.667532ms iteration = 17 status = {}
95512024-11-14T04:54:19.785ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:40091
95522024-11-14T04:54:19.785ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40091 req_id = 8e39326d-b0c2-449e-9d94-c55e5b598b38 roles = RoleSet { roles: {} } uri = /metrics/producers
95532024-11-14T04:54:19.785ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40091 req_id = 8e39326d-b0c2-449e-9d94-c55e5b598b38 resource = Database result = Ok(()) uri = /metrics/producers
95542024-11-14T04:54:19.822ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = e5586e29-e301-4b24-9a2c-39545da75716
95552024-11-14T04:54:19.822ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36725 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40091 req_id = 8e39326d-b0c2-449e-9d94-c55e5b598b38 response_code = 201 uri = /metrics/producers
95562024-11-14T04:54:20.257ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1046 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = b3340cb7-5d4d-415c-a447-5df5676ea1ec response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
95572024-11-14T04:54:25.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 907b6d2e-50ce-46d6-aa58-828c6dd8dea2 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
95582024-11-14T04:54:25.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 907b6d2e-50ce-46d6-aa58-828c6dd8dea2 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
95592024-11-14T04:54:25.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2604 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 907b6d2e-50ce-46d6-aa58-828c6dd8dea2 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
95602024-11-14T04:54:25.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = ece50ed4-bb3f-4b93-85d6-0d962ced6b4c roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
95612024-11-14T04:54:25.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = ece50ed4-bb3f-4b93-85d6-0d962ced6b4c resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
95622024-11-14T04:54:25.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1862 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = ece50ed4-bb3f-4b93-85d6-0d962ced6b4c response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
95632024-11-14T04:54:25.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
95642024-11-14T04:54:25.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
95652024-11-14T04:54:25.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95662024-11-14T04:54:25.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95672024-11-14T04:54:25.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95682024-11-14T04:54:25.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95692024-11-14T04:54:25.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95702024-11-14T04:54:25.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
95712024-11-14T04:54:25.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95722024-11-14T04:54:25.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95732024-11-14T04:54:25.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95742024-11-14T04:54:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95752024-11-14T04:54:25.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95762024-11-14T04:54:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
95772024-11-14T04:54:25.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95782024-11-14T04:54:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95792024-11-14T04:54:25.343ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
95802024-11-14T04:54:25.343ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.252ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
95812024-11-14T04:54:25.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95822024-11-14T04:54:25.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95832024-11-14T04:54:25.355ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.655979ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
95842024-11-14T04:54:30.803ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:53055
95852024-11-14T04:54:30.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53055 req_id = e6dc5f8b-9f73-4141-b55c-fbd96e90e0bd roles = RoleSet { roles: {} } uri = /metrics/producers
95862024-11-14T04:54:30.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53055 req_id = e6dc5f8b-9f73-4141-b55c-fbd96e90e0bd resource = Database result = Ok(()) uri = /metrics/producers
95872024-11-14T04:54:30.844ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = f4c8f330-91d6-4976-b637-c72f900d8c87
95882024-11-14T04:54:30.844ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41617 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53055 req_id = e6dc5f8b-9f73-4141-b55c-fbd96e90e0bd response_code = 201 uri = /metrics/producers
95892024-11-14T04:54:32.697ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:45764
95902024-11-14T04:54:32.697ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45764 req_id = 9e16cd21-d5f5-46a1-8bfc-09b03407c3e4 roles = RoleSet { roles: {} } uri = /metrics/producers
95912024-11-14T04:54:32.697ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45764 req_id = 9e16cd21-d5f5-46a1-8bfc-09b03407c3e4 resource = Database result = Ok(()) uri = /metrics/producers
95922024-11-14T04:54:32.707ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 18d4ee8d-1d67-44aa-96f9-89dfcadba34d
95932024-11-14T04:54:32.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10168 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45764 req_id = 9e16cd21-d5f5-46a1-8bfc-09b03407c3e4 response_code = 201 uri = /metrics/producers
95942024-11-14T04:54:35.258ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1398 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = d5f93a8f-5dc9-48e1-84a5-c16712784a2c response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
95952024-11-14T04:54:35.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
95962024-11-14T04:54:35.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95972024-11-14T04:54:35.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
95982024-11-14T04:54:35.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95992024-11-14T04:54:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96002024-11-14T04:54:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96012024-11-14T04:54:35.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96022024-11-14T04:54:35.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96032024-11-14T04:54:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96042024-11-14T04:54:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
96052024-11-14T04:54:35.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96062024-11-14T04:54:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96072024-11-14T04:54:35.321ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.47907ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
96082024-11-14T04:54:35.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96092024-11-14T04:54:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96102024-11-14T04:54:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96112024-11-14T04:54:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
96122024-11-14T04:54:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96132024-11-14T04:54:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96142024-11-14T04:54:35.337ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
96152024-11-14T04:54:35.337ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.948367ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
96162024-11-14T04:54:40.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 61b36099-0572-49e1-9cee-978f1b635aa1 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
96172024-11-14T04:54:40.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 61b36099-0572-49e1-9cee-978f1b635aa1 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
96182024-11-14T04:54:40.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3278 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 61b36099-0572-49e1-9cee-978f1b635aa1 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
96192024-11-14T04:54:40.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = cdc114d5-4511-4779-8d45-28d0db77e031 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
96202024-11-14T04:54:40.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = cdc114d5-4511-4779-8d45-28d0db77e031 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
96212024-11-14T04:54:40.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2580 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = cdc114d5-4511-4779-8d45-28d0db77e031 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
96222024-11-14T04:54:44.491ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:50140
96232024-11-14T04:54:44.491ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:50140 req_id = 430e4106-8e9d-4a23-8c35-7b781dca6c0a roles = RoleSet { roles: {} } uri = /metrics/producers
96242024-11-14T04:54:44.491ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:50140 req_id = 430e4106-8e9d-4a23-8c35-7b781dca6c0a resource = Database result = Ok(()) uri = /metrics/producers
96252024-11-14T04:54:44.504ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = bc37b1eb-3677-438e-a276-b6d3e6fe3f8e
96262024-11-14T04:54:44.504ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 13340 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:50140 req_id = 430e4106-8e9d-4a23-8c35-7b781dca6c0a response_code = 201 uri = /metrics/producers
96272024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
96282024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
96292024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
96302024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96312024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
96322024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
96332024-11-14T04:54:45.317ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
96342024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
96352024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96362024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
96372024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
96382024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96392024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96402024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
96412024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
96422024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
96432024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
96442024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
96452024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
96462024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96472024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
96482024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
96492024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
96502024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
96512024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96522024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
96532024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
96542024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
96552024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
96562024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
96572024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
96582024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
96592024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
96602024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
96612024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
96622024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
96632024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96642024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96652024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96662024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
96672024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
96682024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96692024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
96702024-11-14T04:54:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
96712024-11-14T04:54:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96722024-11-14T04:54:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96732024-11-14T04:54:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96742024-11-14T04:54:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
96752024-11-14T04:54:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
96762024-11-14T04:54:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
96772024-11-14T04:54:45.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
96782024-11-14T04:54:45.324ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.723414ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
96792024-11-14T04:54:45.324ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1464, tv_nsec: 100048033 } })
96802024-11-14T04:54:45.324ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
96812024-11-14T04:54:45.325ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1464, tv_nsec: 100668761 } })
96822024-11-14T04:54:45.325ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
96832024-11-14T04:54:45.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96842024-11-14T04:54:45.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96852024-11-14T04:54:45.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
96862024-11-14T04:54:45.328ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96872024-11-14T04:54:45.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96882024-11-14T04:54:45.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
96892024-11-14T04:54:45.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96902024-11-14T04:54:45.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96912024-11-14T04:54:45.331ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96922024-11-14T04:54:45.332ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96932024-11-14T04:54:45.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
96942024-11-14T04:54:45.334ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96952024-11-14T04:54:45.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96962024-11-14T04:54:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
96972024-11-14T04:54:45.335ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
96982024-11-14T04:54:45.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
96992024-11-14T04:54:45.339ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97002024-11-14T04:54:45.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
97012024-11-14T04:54:45.340ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97022024-11-14T04:54:45.341ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
97032024-11-14T04:54:45.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
97042024-11-14T04:54:45.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
97052024-11-14T04:54:45.343ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97062024-11-14T04:54:45.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97072024-11-14T04:54:45.346ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
97082024-11-14T04:54:45.346ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97092024-11-14T04:54:45.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
97102024-11-14T04:54:45.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97112024-11-14T04:54:45.348ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
97122024-11-14T04:54:45.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97132024-11-14T04:54:45.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
97142024-11-14T04:54:45.349ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
97152024-11-14T04:54:45.349ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97162024-11-14T04:54:45.350ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
97172024-11-14T04:54:45.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97182024-11-14T04:54:45.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
97192024-11-14T04:54:45.353ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 35.665102ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
97202024-11-14T04:54:45.353ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1464, tv_nsec: 129445975 } })
97212024-11-14T04:54:45.354ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
97222024-11-14T04:54:45.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 39.728131ms iteration = 18 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
97232024-11-14T04:54:45.360ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97242024-11-14T04:54:45.361ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
97252024-11-14T04:54:45.361ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 43.709416ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
97262024-11-14T04:54:45.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
97272024-11-14T04:54:45.378ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97282024-11-14T04:54:45.380ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
97292024-11-14T04:54:45.380ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97302024-11-14T04:54:45.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
97312024-11-14T04:54:45.383ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97322024-11-14T04:54:45.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
97332024-11-14T04:54:45.386ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97342024-11-14T04:54:45.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97352024-11-14T04:54:45.389ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97362024-11-14T04:54:45.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97372024-11-14T04:54:45.392ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97382024-11-14T04:54:45.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
97392024-11-14T04:54:45.394ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97402024-11-14T04:54:45.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
97412024-11-14T04:54:45.396ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97422024-11-14T04:54:45.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
97432024-11-14T04:54:45.399ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97442024-11-14T04:54:45.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97452024-11-14T04:54:45.405ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 87.861204ms iteration = 18 status = {"errors":[],"records_set_to_done":[]}
97462024-11-14T04:54:45.405ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 88.246296ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
97472024-11-14T04:54:45.407ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
97482024-11-14T04:54:45.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
97492024-11-14T04:54:45.413ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97502024-11-14T04:54:45.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97512024-11-14T04:54:45.415ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 98.550064ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
97522024-11-14T04:54:45.418ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97532024-11-14T04:54:45.418ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
97542024-11-14T04:54:45.418ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
97552024-11-14T04:54:45.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
97562024-11-14T04:54:45.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97572024-11-14T04:54:45.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
97582024-11-14T04:54:45.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97592024-11-14T04:54:45.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97602024-11-14T04:54:45.468ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 150.90515ms iteration = 20 status = {"physical_disks_added":0}
97612024-11-14T04:54:45.481ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
97622024-11-14T04:54:45.481ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
97632024-11-14T04:54:45.481ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2080adda-2ed5-4921-8a0e-ab3ddde7fdad", "content-length": "26", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
97642024-11-14T04:54:45.481ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
97652024-11-14T04:54:45.484ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
97662024-11-14T04:54:45.484ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
97672024-11-14T04:54:45.484ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
97682024-11-14T04:54:45.484ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bfe003a-49a4-4519-abc1-49f733fb6b6c", "content-length": "26", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
97692024-11-14T04:54:45.484ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
97702024-11-14T04:54:45.486ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
97712024-11-14T04:54:45.518ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
97722024-11-14T04:54:45.518ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
97732024-11-14T04:54:45.518ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a203188e-3189-4065-8a11-8bfba22aa89d", "content-length": "26", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
97742024-11-14T04:54:45.518ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
97752024-11-14T04:54:45.520ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
97762024-11-14T04:54:45.576ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
97772024-11-14T04:54:45.576ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
97782024-11-14T04:54:45.577ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "06781725-60bf-4c77-af81-fd0746f92a83", "content-length": "372", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
97792024-11-14T04:54:45.577ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97802024-11-14T04:54:45.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97812024-11-14T04:54:45.610ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
97822024-11-14T04:54:45.611ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b78a232d-5de7-4e97-bd1c-ccf66040cf86", "content-length": "1", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
97832024-11-14T04:54:45.614ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
97842024-11-14T04:54:45.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
97852024-11-14T04:54:45.615ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 297.972889ms iteration = 18 status = {}
97862024-11-14T04:54:45.616ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
97872024-11-14T04:54:45.616ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "64e2cddf-77fe-471f-9e3c-24d4f0c9d5df", "content-length": "691", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
97882024-11-14T04:54:45.616ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
97892024-11-14T04:54:45.616ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
97902024-11-14T04:54:45.616ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
97912024-11-14T04:54:45.617ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 300.715968ms iteration = 18 status = {}
97922024-11-14T04:54:45.618ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
97932024-11-14T04:54:45.618ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97942024-11-14T04:54:45.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
97952024-11-14T04:54:45.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97962024-11-14T04:54:45.620ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97972024-11-14T04:54:45.620ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
97982024-11-14T04:54:45.620ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
97992024-11-14T04:54:45.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
98002024-11-14T04:54:45.624ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98012024-11-14T04:54:45.626ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 309.429161ms iteration = 18 status = null
98022024-11-14T04:54:45.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
98032024-11-14T04:54:45.627ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
98042024-11-14T04:54:45.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
98052024-11-14T04:54:45.629ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 312.037116ms iteration = 18 status = 0
98062024-11-14T04:54:45.635ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
98072024-11-14T04:54:45.635ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
98082024-11-14T04:54:45.635ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "502031d9-cfda-4755-8734-c6db496b068c", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
98092024-11-14T04:54:45.635ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98102024-11-14T04:54:45.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98112024-11-14T04:54:45.636ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
98122024-11-14T04:54:45.636ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 319.702028ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
98132024-11-14T04:54:45.638ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98142024-11-14T04:54:45.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98152024-11-14T04:54:45.641ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98162024-11-14T04:54:45.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98172024-11-14T04:54:45.644ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
98182024-11-14T04:54:45.644ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
98192024-11-14T04:54:45.644ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0d0c3412-6835-4ef5-a250-fbf46b8805c9", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
98202024-11-14T04:54:45.646ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 328.981667ms iteration = 19 status = {}
98212024-11-14T04:54:45.653ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 336.208496ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
98222024-11-14T04:54:45.657ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
98232024-11-14T04:54:45.657ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
98242024-11-14T04:54:45.657ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 339.908513ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
98252024-11-14T04:54:45.684ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98262024-11-14T04:54:45.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98272024-11-14T04:54:45.686ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
98282024-11-14T04:54:45.686ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98292024-11-14T04:54:45.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98302024-11-14T04:54:45.695ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
98312024-11-14T04:54:45.695ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ef77524-b728-445b-af2e-d11a62104711", "content-length": "134", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
98322024-11-14T04:54:45.695ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
98332024-11-14T04:54:45.695ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
98342024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
98352024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
98362024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98372024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98382024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98392024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
98402024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98412024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
98422024-11-14T04:54:45.696ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
98432024-11-14T04:54:45.696ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98442024-11-14T04:54:45.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98452024-11-14T04:54:45.698ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
98462024-11-14T04:54:45.698ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "77923858-e7af-4795-965f-d6b9698fbb3a", "content-length": "2", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
98472024-11-14T04:54:45.698ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
98482024-11-14T04:54:45.698ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "83d1224c-3413-47ed-9993-f0105ed136ab", "content-length": "51", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
98492024-11-14T04:54:45.698ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
98502024-11-14T04:54:45.698ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
98512024-11-14T04:54:45.979ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "827f463c-aa76-4452-ab81-5661f8e5fcb9", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
98522024-11-14T04:54:45.979ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
98532024-11-14T04:54:45.979ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
98542024-11-14T04:54:45.979ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4de129a0-6546-47e5-a5de-bad11d95a411", "content-length": "682", "date": "Thu, 14 Nov 2024 04:54:45 GMT"} })
98552024-11-14T04:54:45.979ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98562024-11-14T04:54:45.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98572024-11-14T04:54:45.983ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98582024-11-14T04:54:45.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98592024-11-14T04:54:45.986ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98602024-11-14T04:54:45.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98612024-11-14T04:54:45.989ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98622024-11-14T04:54:45.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98632024-11-14T04:54:45.995ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 678.516638ms iteration = 18 status = {}
98642024-11-14T04:54:49.189ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:55532
98652024-11-14T04:54:49.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:55532 req_id = ae959173-49ba-425a-bb88-0d62099f5b81 roles = RoleSet { roles: {} } uri = /metrics/producers
98662024-11-14T04:54:49.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:55532 req_id = ae959173-49ba-425a-bb88-0d62099f5b81 resource = Database result = Ok(()) uri = /metrics/producers
98672024-11-14T04:54:49.234ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = c073d51c-abaa-48df-abb6-2f053e9c1bec
98682024-11-14T04:54:49.234ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44893 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:55532 req_id = ae959173-49ba-425a-bb88-0d62099f5b81 response_code = 201 uri = /metrics/producers
98692024-11-14T04:54:50.259ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1239 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = eca24eb9-6c0f-4b94-9622-6c8a26332a22 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
98702024-11-14T04:54:55.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
98712024-11-14T04:54:55.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98722024-11-14T04:54:55.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
98732024-11-14T04:54:55.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98742024-11-14T04:54:55.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98752024-11-14T04:54:55.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98762024-11-14T04:54:55.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98772024-11-14T04:54:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
98782024-11-14T04:54:55.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98792024-11-14T04:54:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98802024-11-14T04:54:55.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98812024-11-14T04:54:55.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98822024-11-14T04:54:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98832024-11-14T04:54:55.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
98842024-11-14T04:54:55.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98852024-11-14T04:54:55.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98862024-11-14T04:54:55.338ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
98872024-11-14T04:54:55.338ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.641016ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
98882024-11-14T04:54:55.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98892024-11-14T04:54:55.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98902024-11-14T04:54:55.382ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.161006ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
98912024-11-14T04:55:05.259ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1615 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 4dc3d414-bc54-4fd5-8436-64782c5f4b82 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
98922024-11-14T04:55:05.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
98932024-11-14T04:55:05.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98942024-11-14T04:55:05.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
98952024-11-14T04:55:05.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98962024-11-14T04:55:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98972024-11-14T04:55:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98982024-11-14T04:55:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98992024-11-14T04:55:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
99002024-11-14T04:55:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99012024-11-14T04:55:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99022024-11-14T04:55:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99032024-11-14T04:55:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99042024-11-14T04:55:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99052024-11-14T04:55:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
99062024-11-14T04:55:05.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99072024-11-14T04:55:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99082024-11-14T04:55:05.339ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
99092024-11-14T04:55:05.339ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.367616ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
99102024-11-14T04:55:05.350ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99112024-11-14T04:55:05.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99122024-11-14T04:55:05.351ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.052295ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
99132024-11-14T04:55:10.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = e8d6d50f-0d73-4c78-8f8b-fa5aeb0e1417 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
99142024-11-14T04:55:10.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = e8d6d50f-0d73-4c78-8f8b-fa5aeb0e1417 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
99152024-11-14T04:55:10.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3283 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = e8d6d50f-0d73-4c78-8f8b-fa5aeb0e1417 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
99162024-11-14T04:55:10.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 2a563d59-c05f-4a50-a9ba-84b223592849 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
99172024-11-14T04:55:10.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 2a563d59-c05f-4a50-a9ba-84b223592849 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
99182024-11-14T04:55:10.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2368 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 2a563d59-c05f-4a50-a9ba-84b223592849 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
99192024-11-14T04:55:11.714ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:57967
99202024-11-14T04:55:11.714ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:57967 req_id = bfbbe7c6-0e37-4450-aa38-39d094e5b8b3 roles = RoleSet { roles: {} } uri = /metrics/producers
99212024-11-14T04:55:11.714ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:57967 req_id = bfbbe7c6-0e37-4450-aa38-39d094e5b8b3 resource = Database result = Ok(()) uri = /metrics/producers
99222024-11-14T04:55:11.726ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = b584ea61-acb3-47a3-a173-0d03f6cae9c9
99232024-11-14T04:55:11.726ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11922 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:57967 req_id = bfbbe7c6-0e37-4450-aa38-39d094e5b8b3 response_code = 201 uri = /metrics/producers
99242024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
99252024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
99262024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99272024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
99282024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
99292024-11-14T04:55:15.316ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
99302024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
99312024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
99322024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.561\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
99332024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
99342024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
99352024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
99362024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99372024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
99382024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
99392024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
99402024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
99412024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
99422024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
99432024-11-14T04:55:15.316ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
99442024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
99452024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
99462024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
99472024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
99482024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
99492024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
99502024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
99512024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
99522024-11-14T04:55:15.316ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:45:15.316583842 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
99532024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
99542024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
99552024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
99562024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
99572024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
99582024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
99592024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
99602024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
99612024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
99622024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
99632024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
99642024-11-14T04:55:15.316ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
99652024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
99662024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99672024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99682024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
99692024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
99702024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
99712024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
99722024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
99732024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
99742024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
99752024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
99762024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99772024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
99782024-11-14T04:55:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
99792024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
99802024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99812024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
99822024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
99832024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
99842024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
99852024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
99862024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
99872024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
99882024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
99892024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
99902024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
99912024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
99922024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
99932024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
99942024-11-14T04:55:15.317ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1494, tv_nsec: 92926013 } })
99952024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
99962024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
99972024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
99982024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99992024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100002024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100012024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
100022024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100032024-11-14T04:55:15.317ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1494, tv_nsec: 93059057 } })
100042024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100052024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
100062024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
100072024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
100082024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
100092024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
100102024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
100112024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
100122024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
100132024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
100142024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
100152024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100162024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100172024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
100182024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
100192024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.03368ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
100202024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
100212024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.262737ms iteration = 11 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
100222024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100232024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100242024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100252024-11-14T04:55:15.317ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100262024-11-14T04:55:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100272024-11-14T04:55:15.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100282024-11-14T04:55:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
100292024-11-14T04:55:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
100302024-11-14T04:55:15.320ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.040987ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
100312024-11-14T04:55:15.320ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
100322024-11-14T04:55:15.320ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
100332024-11-14T04:55:15.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
100342024-11-14T04:55:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
100352024-11-14T04:55:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
100362024-11-14T04:55:15.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100372024-11-14T04:55:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100382024-11-14T04:55:15.324ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100392024-11-14T04:55:15.325ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100402024-11-14T04:55:15.325ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100412024-11-14T04:55:15.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100422024-11-14T04:55:15.326ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1494, tv_nsec: 102313605 } })
100432024-11-14T04:55:15.326ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
100442024-11-14T04:55:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
100452024-11-14T04:55:15.328ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
100462024-11-14T04:55:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
100472024-11-14T04:55:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
100482024-11-14T04:55:15.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
100492024-11-14T04:55:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
100502024-11-14T04:55:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
100512024-11-14T04:55:15.330ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
100522024-11-14T04:55:15.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
100532024-11-14T04:55:15.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
100542024-11-14T04:55:15.332ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
100552024-11-14T04:55:15.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
100562024-11-14T04:55:15.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100572024-11-14T04:55:15.334ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100582024-11-14T04:55:15.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100592024-11-14T04:55:15.335ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1494, tv_nsec: 110902193 } })
100602024-11-14T04:55:15.335ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100612024-11-14T04:55:15.335ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
100622024-11-14T04:55:15.337ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:45:15.316583842 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
100632024-11-14T04:55:15.337ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 20.512654ms iteration = 10 status = {"expiration":"2024-11-14T04:45:15.316583842Z","pruned":[]}
100642024-11-14T04:55:15.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100652024-11-14T04:55:15.338ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100662024-11-14T04:55:15.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100672024-11-14T04:55:15.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
100682024-11-14T04:55:15.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100692024-11-14T04:55:15.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100702024-11-14T04:55:15.340ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100712024-11-14T04:55:15.341ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100722024-11-14T04:55:15.343ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
100732024-11-14T04:55:15.343ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100742024-11-14T04:55:15.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
100752024-11-14T04:55:15.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
100762024-11-14T04:55:15.344ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
100772024-11-14T04:55:15.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
100782024-11-14T04:55:15.351ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
100792024-11-14T04:55:15.351ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
100802024-11-14T04:55:15.351ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 34.247462ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
100812024-11-14T04:55:15.351ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 34.961343ms iteration = 19 status = {"errors":[],"records_set_to_done":[]}
100822024-11-14T04:55:15.361ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 43.998104ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
100832024-11-14T04:55:15.368ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100842024-11-14T04:55:15.369ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
100852024-11-14T04:55:15.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
100862024-11-14T04:55:15.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100872024-11-14T04:55:15.375ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100882024-11-14T04:55:15.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100892024-11-14T04:55:15.375ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1494, tv_nsec: 151572891 } })
100902024-11-14T04:55:15.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100912024-11-14T04:55:15.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100922024-11-14T04:55:15.376ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
100932024-11-14T04:55:15.380ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
100942024-11-14T04:55:15.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
100952024-11-14T04:55:15.388ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100962024-11-14T04:55:15.391ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
100972024-11-14T04:55:15.391ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100982024-11-14T04:55:15.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100992024-11-14T04:55:15.391ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101002024-11-14T04:55:15.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
101012024-11-14T04:55:15.402ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
101022024-11-14T04:55:15.402ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 85.588168ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
101032024-11-14T04:55:15.406ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
101042024-11-14T04:55:15.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
101052024-11-14T04:55:15.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101062024-11-14T04:55:15.411ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101072024-11-14T04:55:15.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101082024-11-14T04:55:15.413ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
101092024-11-14T04:55:15.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101102024-11-14T04:55:15.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101112024-11-14T04:55:15.429ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
101122024-11-14T04:55:15.429ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
101132024-11-14T04:55:15.429ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
101142024-11-14T04:55:15.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
101152024-11-14T04:55:15.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
101162024-11-14T04:55:15.432ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 115.359981ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
101172024-11-14T04:55:15.435ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101182024-11-14T04:55:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
101192024-11-14T04:55:15.440ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101202024-11-14T04:55:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
101212024-11-14T04:55:15.443ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 126.543685ms iteration = 19 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
101222024-11-14T04:55:15.445ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101232024-11-14T04:55:15.448ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
101242024-11-14T04:55:15.449ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
101252024-11-14T04:55:15.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
101262024-11-14T04:55:15.451ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101272024-11-14T04:55:15.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
101282024-11-14T04:55:15.454ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
101292024-11-14T04:55:15.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
101302024-11-14T04:55:15.467ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
101312024-11-14T04:55:15.467ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
101322024-11-14T04:55:15.467ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 151.25005ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
101332024-11-14T04:55:15.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101342024-11-14T04:55:15.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
101352024-11-14T04:55:15.475ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101362024-11-14T04:55:15.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
101372024-11-14T04:55:15.483ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
101382024-11-14T04:55:15.483ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
101392024-11-14T04:55:15.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
101402024-11-14T04:55:15.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
101412024-11-14T04:55:15.485ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101422024-11-14T04:55:15.486ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101432024-11-14T04:55:15.486ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
101442024-11-14T04:55:15.486ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 170.291301ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
101452024-11-14T04:55:15.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
101462024-11-14T04:55:15.488ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
101472024-11-14T04:55:15.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
101482024-11-14T04:55:15.488ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 172.519366ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
101492024-11-14T04:55:15.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
101502024-11-14T04:55:15.489ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101512024-11-14T04:55:15.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101522024-11-14T04:55:15.491ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 174.908685ms iteration = 5 status = {"nsuccess":5}
101532024-11-14T04:55:15.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
101542024-11-14T04:55:15.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
101552024-11-14T04:55:15.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
101562024-11-14T04:55:15.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101572024-11-14T04:55:15.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
101582024-11-14T04:55:15.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101592024-11-14T04:55:15.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101602024-11-14T04:55:15.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101612024-11-14T04:55:15.514ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101622024-11-14T04:55:15.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
101632024-11-14T04:55:15.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101642024-11-14T04:55:15.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101652024-11-14T04:55:15.521ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 205.090479ms iteration = 19 status = 0
101662024-11-14T04:55:15.522ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101672024-11-14T04:55:15.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
101682024-11-14T04:55:15.524ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101692024-11-14T04:55:15.527ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
101702024-11-14T04:55:15.527ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101712024-11-14T04:55:15.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
101722024-11-14T04:55:15.549ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
101732024-11-14T04:55:15.549ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
101742024-11-14T04:55:15.549ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
101752024-11-14T04:55:15.549ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
101762024-11-14T04:55:15.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
101772024-11-14T04:55:15.552ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
101782024-11-14T04:55:15.552ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
101792024-11-14T04:55:15.552ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
101802024-11-14T04:55:15.552ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101812024-11-14T04:55:15.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101822024-11-14T04:55:15.554ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 237.611721ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
101832024-11-14T04:55:15.554ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101842024-11-14T04:55:15.555ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
101852024-11-14T04:55:15.555ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
101862024-11-14T04:55:15.555ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "185edd05-070e-4b3e-98d4-34c6ef954592", "content-length": "26", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
101872024-11-14T04:55:15.555ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
101882024-11-14T04:55:15.555ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 238.902959ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
101892024-11-14T04:55:15.557ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
101902024-11-14T04:55:15.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
101912024-11-14T04:55:15.559ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101922024-11-14T04:55:15.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101932024-11-14T04:55:15.561ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
101942024-11-14T04:55:15.561ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
101952024-11-14T04:55:15.561ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 244.941974ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
101962024-11-14T04:55:15.561ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
101972024-11-14T04:55:15.561ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
101982024-11-14T04:55:15.561ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101992024-11-14T04:55:15.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
102002024-11-14T04:55:15.565ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102012024-11-14T04:55:15.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
102022024-11-14T04:55:15.569ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
102032024-11-14T04:55:15.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
102042024-11-14T04:55:15.572ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
102052024-11-14T04:55:15.572ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
102062024-11-14T04:55:15.572ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
102072024-11-14T04:55:15.573ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 256.323532ms iteration = 11 status = {"generation":2}
102082024-11-14T04:55:15.589ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
102092024-11-14T04:55:15.589ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
102102024-11-14T04:55:15.589ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "95397342-5400-4ab9-8e3a-e3c132273150", "content-length": "26", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
102112024-11-14T04:55:15.589ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
102122024-11-14T04:55:15.590ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102132024-11-14T04:55:15.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102142024-11-14T04:55:15.595ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
102152024-11-14T04:55:15.598ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102162024-11-14T04:55:15.602ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
102172024-11-14T04:55:15.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
102182024-11-14T04:55:15.603ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102192024-11-14T04:55:15.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102202024-11-14T04:55:15.605ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
102212024-11-14T04:55:15.606ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
102222024-11-14T04:55:15.606ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102232024-11-14T04:55:15.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102242024-11-14T04:55:15.608ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102252024-11-14T04:55:15.608ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
102262024-11-14T04:55:15.608ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 291.508142ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
102272024-11-14T04:55:15.610ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
102282024-11-14T04:55:15.610ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
102292024-11-14T04:55:15.610ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5420b0a-15b0-4f83-9280-cfd21e24b784", "content-length": "26", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
102302024-11-14T04:55:15.610ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
102312024-11-14T04:55:15.611ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
102322024-11-14T04:55:15.611ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102332024-11-14T04:55:15.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102342024-11-14T04:55:15.612ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
102352024-11-14T04:55:15.614ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
102362024-11-14T04:55:15.615ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
102372024-11-14T04:55:15.615ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
102382024-11-14T04:55:15.615ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 298.560406ms iteration = 11 status = {"generation":1}
102392024-11-14T04:55:15.647ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
102402024-11-14T04:55:15.655ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102412024-11-14T04:55:15.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102422024-11-14T04:55:15.668ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
102432024-11-14T04:55:15.668ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 351.446997ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
102442024-11-14T04:55:15.675ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 358.980755ms iteration = 21 status = {"physical_disks_added":0}
102452024-11-14T04:55:15.695ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102462024-11-14T04:55:15.697ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
102472024-11-14T04:55:15.697ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102482024-11-14T04:55:15.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102492024-11-14T04:55:15.699ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102502024-11-14T04:55:15.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
102512024-11-14T04:55:15.702ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102522024-11-14T04:55:15.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102532024-11-14T04:55:15.708ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
102542024-11-14T04:55:15.708ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
102552024-11-14T04:55:15.708ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc8187c9-4957-40a7-b9c8-43e74f8e9200", "content-length": "372", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
102562024-11-14T04:55:15.708ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102572024-11-14T04:55:15.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102582024-11-14T04:55:15.744ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 427.552592ms iteration = 19 status = {}
102592024-11-14T04:55:15.761ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
102602024-11-14T04:55:15.761ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
102612024-11-14T04:55:15.761ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e21951f4-db6c-4226-9f0f-9d405309a106", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
102622024-11-14T04:55:15.761ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102632024-11-14T04:55:15.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102642024-11-14T04:55:15.763ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102652024-11-14T04:55:15.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102662024-11-14T04:55:15.767ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102672024-11-14T04:55:15.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102682024-11-14T04:55:15.771ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
102692024-11-14T04:55:15.772ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
102702024-11-14T04:55:15.772ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5012ee0c-364e-47ea-8e01-d3a1d73d1f7e", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
102712024-11-14T04:55:15.775ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 459.26225ms iteration = 20 status = {}
102722024-11-14T04:55:15.814ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
102732024-11-14T04:55:15.815ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "d699b741-3d93-477f-ba51-4941402d9ddb", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
102742024-11-14T04:55:15.815ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
102752024-11-14T04:55:15.817ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "526e679a-a58e-4640-b28e-30d471150ae9", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
102762024-11-14T04:55:15.817ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
102772024-11-14T04:55:15.820ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "98f152be-2ff2-4bc2-a42a-e30b0f63fe92", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
102782024-11-14T04:55:15.820ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
102792024-11-14T04:55:15.826ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
102802024-11-14T04:55:15.826ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 509.894706ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
102812024-11-14T04:55:15.853ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
102822024-11-14T04:55:15.853ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "929b544d-c873-46b0-85f6-9c52746bb0c0", "content-length": "1", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
102832024-11-14T04:55:15.856ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
102842024-11-14T04:55:15.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
102852024-11-14T04:55:15.865ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
102862024-11-14T04:55:15.865ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e4a3aff2-2eb8-4ed8-be7e-cdbae4ae27ac", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
102872024-11-14T04:55:15.865ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
102882024-11-14T04:55:15.867ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9b063803-31d5-4b16-b730-27e7e553585b", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
102892024-11-14T04:55:15.867ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
102902024-11-14T04:55:15.869ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
102912024-11-14T04:55:15.869ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 552.952593ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
102922024-11-14T04:55:15.908ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
102932024-11-14T04:55:15.911ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 594.428424ms iteration = 19 status = null
102942024-11-14T04:55:15.974ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102952024-11-14T04:55:15.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102962024-11-14T04:55:15.976ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
102972024-11-14T04:55:15.976ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102982024-11-14T04:55:15.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102992024-11-14T04:55:15.986ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
103002024-11-14T04:55:15.986ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3289a78-a39d-4b10-9adc-47b485c93622", "content-length": "134", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
103012024-11-14T04:55:15.986ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
103022024-11-14T04:55:15.986ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
103032024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
103042024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
103052024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
103062024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
103072024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
103082024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
103092024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
103102024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
103112024-11-14T04:55:15.987ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
103122024-11-14T04:55:15.987ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103132024-11-14T04:55:15.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103142024-11-14T04:55:15.989ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
103152024-11-14T04:55:15.990ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "2de008c1-b196-4380-979a-16db876098e0", "content-length": "2", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
103162024-11-14T04:55:15.990ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
103172024-11-14T04:55:15.990ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b708ec49-dc1f-4fa0-858e-883b38f21d9a", "content-length": "51", "date": "Thu, 14 Nov 2024 04:55:15 GMT"} })
103182024-11-14T04:55:15.990ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
103192024-11-14T04:55:15.990ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
103202024-11-14T04:55:16.008ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c94e5533-333a-41d0-bf28-53fbfafb634a", "content-length": "691", "date": "Thu, 14 Nov 2024 04:55:16 GMT"} })
103212024-11-14T04:55:16.008ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
103222024-11-14T04:55:16.008ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
103232024-11-14T04:55:16.008ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
103242024-11-14T04:55:16.010ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 693.72987ms iteration = 19 status = {}
103252024-11-14T04:55:16.093ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5b82bbab-830a-451d-8da4-142301a5f2d9", "date": "Thu, 14 Nov 2024 04:55:16 GMT"} })
103262024-11-14T04:55:16.093ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
103272024-11-14T04:55:16.093ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
103282024-11-14T04:55:16.096ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb154e5b-4ff5-427d-8396-f99e9c445441", "content-length": "682", "date": "Thu, 14 Nov 2024 04:55:16 GMT"} })
103292024-11-14T04:55:16.096ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103302024-11-14T04:55:16.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103312024-11-14T04:55:16.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103322024-11-14T04:55:16.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103332024-11-14T04:55:16.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103342024-11-14T04:55:16.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103352024-11-14T04:55:16.105ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103362024-11-14T04:55:16.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103372024-11-14T04:55:16.113ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 796.686051ms iteration = 19 status = {}
103382024-11-14T04:55:20.259ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1223 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 391862fe-d620-4233-a4a8-307b9abc30b4 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
103392024-11-14T04:55:25.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 3d654369-7122-4a5a-be7e-a43ed87b7479 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
103402024-11-14T04:55:25.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 3d654369-7122-4a5a-be7e-a43ed87b7479 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
103412024-11-14T04:55:25.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3487 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 3d654369-7122-4a5a-be7e-a43ed87b7479 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
103422024-11-14T04:55:25.253ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 00d3899a-773d-4c42-937b-db8462d31620 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
103432024-11-14T04:55:25.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 00d3899a-773d-4c42-937b-db8462d31620 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
103442024-11-14T04:55:25.256ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2432 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 00d3899a-773d-4c42-937b-db8462d31620 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
103452024-11-14T04:55:25.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
103462024-11-14T04:55:25.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
103472024-11-14T04:55:25.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103482024-11-14T04:55:25.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103492024-11-14T04:55:25.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103502024-11-14T04:55:25.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103512024-11-14T04:55:25.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103522024-11-14T04:55:25.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
103532024-11-14T04:55:25.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103542024-11-14T04:55:25.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103552024-11-14T04:55:25.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103562024-11-14T04:55:25.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103572024-11-14T04:55:25.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103582024-11-14T04:55:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
103592024-11-14T04:55:25.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103602024-11-14T04:55:25.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103612024-11-14T04:55:25.337ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
103622024-11-14T04:55:25.337ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.06428ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
103632024-11-14T04:55:25.351ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103642024-11-14T04:55:25.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103652024-11-14T04:55:25.380ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.395004ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
103662024-11-14T04:55:35.258ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1044 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 70e6e5ad-7f13-41c8-95cf-ab3e690b0da1 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
103672024-11-14T04:55:35.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
103682024-11-14T04:55:35.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
103692024-11-14T04:55:35.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103702024-11-14T04:55:35.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103712024-11-14T04:55:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103722024-11-14T04:55:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103732024-11-14T04:55:35.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103742024-11-14T04:55:35.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103752024-11-14T04:55:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103762024-11-14T04:55:35.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.123919ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
103772024-11-14T04:55:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
103782024-11-14T04:55:35.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103792024-11-14T04:55:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103802024-11-14T04:55:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103812024-11-14T04:55:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103822024-11-14T04:55:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103832024-11-14T04:55:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
103842024-11-14T04:55:35.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103852024-11-14T04:55:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103862024-11-14T04:55:35.335ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
103872024-11-14T04:55:35.335ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.6953ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
103882024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
103892024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
103902024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
103912024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103922024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
103932024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
103942024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103952024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
103962024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
103972024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
103982024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
103992024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
104002024-11-14T04:55:45.316ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
104012024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
104022024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104032024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
104042024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
104052024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
104062024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104072024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
104082024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
104092024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
104102024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
104112024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
104122024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
104132024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
104142024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
104152024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
104162024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104172024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104182024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
104192024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
104202024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
104212024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
104222024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
104232024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104242024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
104252024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
104262024-11-14T04:55:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
104272024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
104282024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
104292024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
104302024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
104312024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
104322024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
104332024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
104342024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
104352024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
104362024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
104372024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
104382024-11-14T04:55:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
104392024-11-14T04:55:45.319ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.399429ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
104402024-11-14T04:55:45.319ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 95788712 } })
104412024-11-14T04:55:45.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104422024-11-14T04:55:45.320ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
104432024-11-14T04:55:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104442024-11-14T04:55:45.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
104452024-11-14T04:55:45.322ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104462024-11-14T04:55:45.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104472024-11-14T04:55:45.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
104482024-11-14T04:55:45.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
104492024-11-14T04:55:45.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
104502024-11-14T04:55:45.324ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104512024-11-14T04:55:45.325ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 101832007 } })
104522024-11-14T04:55:45.326ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
104532024-11-14T04:55:45.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
104542024-11-14T04:55:45.326ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104552024-11-14T04:55:45.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104562024-11-14T04:55:45.329ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
104572024-11-14T04:55:45.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
104582024-11-14T04:55:45.337ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 20.449542ms iteration = 20 status = {"errors":[],"records_set_to_done":[]}
104592024-11-14T04:55:45.338ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 114363230 } })
104602024-11-14T04:55:45.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104612024-11-14T04:55:45.338ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
104622024-11-14T04:55:45.340ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
104632024-11-14T04:55:45.340ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104642024-11-14T04:55:45.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104652024-11-14T04:55:45.369ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104662024-11-14T04:55:45.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
104672024-11-14T04:55:45.374ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
104682024-11-14T04:55:45.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
104692024-11-14T04:55:45.375ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104702024-11-14T04:55:45.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
104712024-11-14T04:55:45.378ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
104722024-11-14T04:55:45.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
104732024-11-14T04:55:45.381ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
104742024-11-14T04:55:45.381ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 64.976642ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
104752024-11-14T04:55:45.381ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
104762024-11-14T04:55:45.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
104772024-11-14T04:55:45.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104782024-11-14T04:55:45.385ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
104792024-11-14T04:55:45.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
104802024-11-14T04:55:45.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
104812024-11-14T04:55:45.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104822024-11-14T04:55:45.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104832024-11-14T04:55:45.395ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
104842024-11-14T04:55:45.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
104852024-11-14T04:55:45.396ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 79.06259ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
104862024-11-14T04:55:45.401ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
104872024-11-14T04:55:45.401ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
104882024-11-14T04:55:45.401ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
104892024-11-14T04:55:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
104902024-11-14T04:55:45.402ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 84.94679ms iteration = 20 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
104912024-11-14T04:55:45.462ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
104922024-11-14T04:55:45.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
104932024-11-14T04:55:45.465ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
104942024-11-14T04:55:45.465ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
104952024-11-14T04:55:45.465ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "35a68048-8983-4904-b1cb-5d7815c42157", "content-length": "26", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
104962024-11-14T04:55:45.465ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
104972024-11-14T04:55:45.467ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
104982024-11-14T04:55:45.594ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
104992024-11-14T04:55:45.595ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4e2efa7-fc27-422d-83f1-657afbae9ecd", "content-length": "1", "date": "Thu, 14 Nov 2024 04:55:44 GMT"} })
105002024-11-14T04:55:45.596ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
105012024-11-14T04:55:45.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
105022024-11-14T04:55:45.631ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
105032024-11-14T04:55:45.631ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
105042024-11-14T04:55:45.631ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e76e8d6-0fb3-447b-bb12-46c9b2fd2cc3", "content-length": "26", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
105052024-11-14T04:55:45.631ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
105062024-11-14T04:55:45.633ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
105072024-11-14T04:55:45.648ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
105082024-11-14T04:55:45.648ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
105092024-11-14T04:55:45.648ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a5d1df4-911b-427c-ac2b-9c3c2dfe2c90", "content-length": "26", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
105102024-11-14T04:55:45.648ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
105112024-11-14T04:55:45.650ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
105122024-11-14T04:55:45.669ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
105132024-11-14T04:55:45.669ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
105142024-11-14T04:55:45.671ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105152024-11-14T04:55:45.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105162024-11-14T04:55:45.672ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
105172024-11-14T04:55:45.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
105182024-11-14T04:55:45.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105192024-11-14T04:55:45.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
105202024-11-14T04:55:45.675ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 359.531082ms iteration = 20 status = null
105212024-11-14T04:55:45.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105222024-11-14T04:55:45.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
105232024-11-14T04:55:45.678ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
105242024-11-14T04:55:45.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
105252024-11-14T04:55:45.705ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 389.010275ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
105262024-11-14T04:55:45.706ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
105272024-11-14T04:55:45.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
105282024-11-14T04:55:45.707ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
105292024-11-14T04:55:45.707ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
105302024-11-14T04:55:45.707ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
105312024-11-14T04:55:45.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
105322024-11-14T04:55:45.708ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105332024-11-14T04:55:45.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105342024-11-14T04:55:45.710ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 394.215216ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
105352024-11-14T04:55:45.710ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105362024-11-14T04:55:45.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105372024-11-14T04:55:45.712ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105382024-11-14T04:55:45.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105392024-11-14T04:55:45.714ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 397.957255ms iteration = 20 status = {}
105402024-11-14T04:55:45.716ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
105412024-11-14T04:55:45.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105422024-11-14T04:55:45.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
105432024-11-14T04:55:45.716ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
105442024-11-14T04:55:45.716ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105452024-11-14T04:55:45.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105462024-11-14T04:55:45.719ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 403.627478ms iteration = 20 status = 0
105472024-11-14T04:55:45.729ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
105482024-11-14T04:55:45.729ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 413.181836ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
105492024-11-14T04:55:45.738ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 420.899648ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
105502024-11-14T04:55:45.744ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
105512024-11-14T04:55:45.744ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
105522024-11-14T04:55:45.744ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 428.349594ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
105532024-11-14T04:55:45.756ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b20c3a5e-c350-4c9d-a441-2c201269f80c", "content-length": "372", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
105542024-11-14T04:55:45.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105552024-11-14T04:55:45.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105562024-11-14T04:55:45.786ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
105572024-11-14T04:55:45.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
105582024-11-14T04:55:45.788ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105592024-11-14T04:55:45.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
105602024-11-14T04:55:45.790ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
105612024-11-14T04:55:45.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
105622024-11-14T04:55:45.793ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 477.117497ms iteration = 22 status = {"physical_disks_added":0}
105632024-11-14T04:55:45.796ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
105642024-11-14T04:55:45.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
105652024-11-14T04:55:45.798ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
105662024-11-14T04:55:45.798ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
105672024-11-14T04:55:45.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
105682024-11-14T04:55:45.808ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
105692024-11-14T04:55:45.808ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "4effb485-d65b-437d-96dd-a6cc18bdb6d4", "content-length": "134", "date": "Thu, 14 Nov 2024 04:55:44 GMT"} })
105702024-11-14T04:55:45.808ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
105712024-11-14T04:55:45.808ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
105722024-11-14T04:55:45.808ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
105732024-11-14T04:55:45.808ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
105742024-11-14T04:55:45.808ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
105752024-11-14T04:55:45.808ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
105762024-11-14T04:55:45.808ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
105772024-11-14T04:55:45.808ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
105782024-11-14T04:55:45.809ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
105792024-11-14T04:55:45.809ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
105802024-11-14T04:55:45.809ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
105812024-11-14T04:55:45.809ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
105822024-11-14T04:55:45.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
105832024-11-14T04:55:45.809ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
105842024-11-14T04:55:45.809ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
105852024-11-14T04:55:45.811ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
105862024-11-14T04:55:45.811ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfa95f26-61d8-4e4c-a8c4-0e282b699c96", "content-length": "2", "date": "Thu, 14 Nov 2024 04:55:44 GMT"} })
105872024-11-14T04:55:45.811ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
105882024-11-14T04:55:45.811ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "efbdc892-ec6b-445c-a608-14cdcac10401", "content-length": "51", "date": "Thu, 14 Nov 2024 04:55:44 GMT"} })
105892024-11-14T04:55:45.811ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
105902024-11-14T04:55:45.811ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
105912024-11-14T04:55:45.813ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2e1b56f8-5b18-4c76-af41-07bcfcce5947", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
105922024-11-14T04:55:45.813ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105932024-11-14T04:55:45.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105942024-11-14T04:55:45.816ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105952024-11-14T04:55:45.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105962024-11-14T04:55:45.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105972024-11-14T04:55:45.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105982024-11-14T04:55:45.820ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105992024-11-14T04:55:45.821ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
106002024-11-14T04:55:45.821ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106012024-11-14T04:55:45.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106022024-11-14T04:55:45.822ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
106032024-11-14T04:55:45.822ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
106042024-11-14T04:55:45.822ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5576f508-bd87-4244-8eb9-1c01b770f141", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
106052024-11-14T04:55:45.824ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 508.041321ms iteration = 21 status = {}
106062024-11-14T04:55:45.907ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
106072024-11-14T04:55:45.908ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e32390d1-d26f-4c8e-b580-2b551eb29656", "content-length": "691", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
106082024-11-14T04:55:45.908ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
106092024-11-14T04:55:45.908ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
106102024-11-14T04:55:45.908ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
106112024-11-14T04:55:45.909ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 593.196157ms iteration = 20 status = {}
106122024-11-14T04:55:45.977ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a3ac7527-3a7e-4f6b-a773-c8c472274279", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
106132024-11-14T04:55:45.977ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
106142024-11-14T04:55:45.977ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
106152024-11-14T04:55:45.977ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ffc1f2a-5047-404e-9e52-00685505be78", "content-length": "682", "date": "Thu, 14 Nov 2024 04:55:45 GMT"} })
106162024-11-14T04:55:45.977ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106172024-11-14T04:55:45.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106182024-11-14T04:55:45.981ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106192024-11-14T04:55:45.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106202024-11-14T04:55:45.984ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106212024-11-14T04:55:45.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106222024-11-14T04:55:45.987ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106232024-11-14T04:55:45.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106242024-11-14T04:55:45.995ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 678.894919ms iteration = 20 status = {}
106252024-11-14T04:55:50.257ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1042 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = d5fc4394-49bd-4845-af54-a44a3c914548 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
106262024-11-14T04:55:55.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
106272024-11-14T04:55:55.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106282024-11-14T04:55:55.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
106292024-11-14T04:55:55.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106302024-11-14T04:55:55.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106312024-11-14T04:55:55.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
106322024-11-14T04:55:55.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106332024-11-14T04:55:55.348ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
106342024-11-14T04:55:55.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106352024-11-14T04:55:55.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106362024-11-14T04:55:55.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106372024-11-14T04:55:55.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106382024-11-14T04:55:55.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106392024-11-14T04:55:55.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106402024-11-14T04:55:55.354ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
106412024-11-14T04:55:55.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106422024-11-14T04:55:55.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106432024-11-14T04:55:55.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
106442024-11-14T04:55:55.365ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
106452024-11-14T04:55:55.365ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.034649ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
106462024-11-14T04:55:55.383ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.244536ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
106472024-11-14T04:56:05.255ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1020 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = e36bab01-06df-43d6-9c63-26a1a8b2333f response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
106482024-11-14T04:56:05.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
106492024-11-14T04:56:05.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106502024-11-14T04:56:05.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
106512024-11-14T04:56:05.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106522024-11-14T04:56:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
106532024-11-14T04:56:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106542024-11-14T04:56:05.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106552024-11-14T04:56:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
106562024-11-14T04:56:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106572024-11-14T04:56:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
106582024-11-14T04:56:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106592024-11-14T04:56:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106602024-11-14T04:56:05.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106612024-11-14T04:56:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106622024-11-14T04:56:05.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106632024-11-14T04:56:05.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
106642024-11-14T04:56:05.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106652024-11-14T04:56:05.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106662024-11-14T04:56:05.337ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
106672024-11-14T04:56:05.337ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.148632ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
106682024-11-14T04:56:05.346ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.519504ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
106692024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
106702024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106712024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
106722024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
106732024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
106742024-11-14T04:56:15.316ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
106752024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 20.331\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
106762024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
106772024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
106782024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
106792024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
106802024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
106812024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
106822024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
106832024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
106842024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
106852024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
106862024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
106872024-11-14T04:56:15.316ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
106882024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106892024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106902024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
106912024-11-14T04:56:15.316ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:46:15.316378286 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
106922024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
106932024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
106942024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106952024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
106962024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
106972024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
106982024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
106992024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
107002024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
107012024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
107022024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
107032024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
107042024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
107052024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
107062024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
107072024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
107082024-11-14T04:56:15.316ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
107092024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
107102024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
107112024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
107122024-11-14T04:56:15.316ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
107132024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
107142024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
107152024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
107162024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
107172024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
107182024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
107192024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Timeout
107202024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
107212024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
107222024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107232024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
107242024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
107252024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
107262024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
107272024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
107282024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
107292024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
107302024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
107312024-11-14T04:56:15.316ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
107322024-11-14T04:56:15.316ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
107332024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
107342024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
107352024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
107362024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107372024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
107382024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
107392024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
107402024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
107412024-11-14T04:56:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
107422024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
107432024-11-14T04:56:15.317ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 93437040 } })
107442024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
107452024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
107462024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107472024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
107482024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107492024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
107502024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
107512024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
107522024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
107532024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
107542024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
107552024-11-14T04:56:15.317ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 93606795 } })
107562024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
107572024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
107582024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
107592024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
107602024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107612024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
107622024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
107632024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
107642024-11-14T04:56:15.317ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
107652024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
107662024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
107672024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
107682024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
107692024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.002509ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
107702024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
107712024-11-14T04:56:15.317ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.399411ms iteration = 12 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
107722024-11-14T04:56:15.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107732024-11-14T04:56:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.182802ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
107742024-11-14T04:56:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
107752024-11-14T04:56:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
107762024-11-14T04:56:15.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
107772024-11-14T04:56:15.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107782024-11-14T04:56:15.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107792024-11-14T04:56:15.322ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
107802024-11-14T04:56:15.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
107812024-11-14T04:56:15.326ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107822024-11-14T04:56:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107832024-11-14T04:56:15.326ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:46:15.316378286 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
107842024-11-14T04:56:15.326ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.273518ms iteration = 11 status = {"expiration":"2024-11-14T04:46:15.316378286Z","pruned":[]}
107852024-11-14T04:56:15.327ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 104052297 } })
107862024-11-14T04:56:15.328ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
107872024-11-14T04:56:15.328ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
107882024-11-14T04:56:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
107892024-11-14T04:56:15.336ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107902024-11-14T04:56:15.336ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107912024-11-14T04:56:15.337ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 113829270 } })
107922024-11-14T04:56:15.337ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
107932024-11-14T04:56:15.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
107942024-11-14T04:56:15.339ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107952024-11-14T04:56:15.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107962024-11-14T04:56:15.343ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
107972024-11-14T04:56:15.343ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
107982024-11-14T04:56:15.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
107992024-11-14T04:56:15.351ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 34.961032ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
108002024-11-14T04:56:15.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
108012024-11-14T04:56:15.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
108022024-11-14T04:56:15.400ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
108032024-11-14T04:56:15.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
108042024-11-14T04:56:15.403ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108052024-11-14T04:56:15.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
108062024-11-14T04:56:15.406ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
108072024-11-14T04:56:15.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
108082024-11-14T04:56:15.413ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108092024-11-14T04:56:15.416ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
108102024-11-14T04:56:15.416ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
108112024-11-14T04:56:15.416ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
108122024-11-14T04:56:15.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
108132024-11-14T04:56:15.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
108142024-11-14T04:56:15.417ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
108152024-11-14T04:56:15.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
108162024-11-14T04:56:15.418ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108172024-11-14T04:56:15.419ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108182024-11-14T04:56:15.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108192024-11-14T04:56:15.420ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108202024-11-14T04:56:15.421ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
108212024-11-14T04:56:15.421ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
108222024-11-14T04:56:15.421ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
108232024-11-14T04:56:15.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
108242024-11-14T04:56:15.421ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108252024-11-14T04:56:15.421ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
108262024-11-14T04:56:15.421ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
108272024-11-14T04:56:15.421ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
108282024-11-14T04:56:15.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
108292024-11-14T04:56:15.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
108302024-11-14T04:56:15.422ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108312024-11-14T04:56:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
108322024-11-14T04:56:15.424ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
108332024-11-14T04:56:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
108342024-11-14T04:56:15.425ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108352024-11-14T04:56:15.426ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 110.337165ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
108362024-11-14T04:56:15.426ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
108372024-11-14T04:56:15.426ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108382024-11-14T04:56:15.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108392024-11-14T04:56:15.427ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 203557369 } })
108402024-11-14T04:56:15.427ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
108412024-11-14T04:56:15.428ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
108422024-11-14T04:56:15.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
108432024-11-14T04:56:15.429ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108442024-11-14T04:56:15.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108452024-11-14T04:56:15.429ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 112.768626ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
108462024-11-14T04:56:15.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108472024-11-14T04:56:15.434ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
108482024-11-14T04:56:15.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
108492024-11-14T04:56:15.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
108502024-11-14T04:56:15.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
108512024-11-14T04:56:15.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
108522024-11-14T04:56:15.435ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
108532024-11-14T04:56:15.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
108542024-11-14T04:56:15.440ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 123.97985ms iteration = 21 status = {"errors":[],"records_set_to_done":[]}
108552024-11-14T04:56:15.442ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
108562024-11-14T04:56:15.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
108572024-11-14T04:56:15.443ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
108582024-11-14T04:56:15.443ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
108592024-11-14T04:56:15.443ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
108602024-11-14T04:56:15.443ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
108612024-11-14T04:56:15.443ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = 61e7c779-5b0c-4739-9b7d-2eea5f2a55fd
108622024-11-14T04:56:15.443ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = d058b523-9a14-427b-91a9-ebb9d4b546a5
108632024-11-14T04:56:15.455ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
108642024-11-14T04:56:15.455ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
108652024-11-14T04:56:15.455ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 139.190781ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
108662024-11-14T04:56:15.456ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = fd45f922-2194-4e21-821b-92d56fca70f7
108672024-11-14T04:56:15.456ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 139.401717ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":3,"recent_failures":[],"recent_recoveries":[]}
108682024-11-14T04:56:15.457ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108692024-11-14T04:56:15.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108702024-11-14T04:56:15.470ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108712024-11-14T04:56:15.473ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
108722024-11-14T04:56:15.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
108732024-11-14T04:56:15.475ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
108742024-11-14T04:56:15.475ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
108752024-11-14T04:56:15.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
108762024-11-14T04:56:15.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108772024-11-14T04:56:15.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
108782024-11-14T04:56:15.496ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
108792024-11-14T04:56:15.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
108802024-11-14T04:56:15.496ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
108812024-11-14T04:56:15.496ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 180.623751ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
108822024-11-14T04:56:15.498ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
108832024-11-14T04:56:15.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
108842024-11-14T04:56:15.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 190.639191ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
108852024-11-14T04:56:15.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
108862024-11-14T04:56:15.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
108872024-11-14T04:56:15.527ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
108882024-11-14T04:56:15.527ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
108892024-11-14T04:56:15.527ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
108902024-11-14T04:56:15.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
108912024-11-14T04:56:15.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
108922024-11-14T04:56:15.529ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
108932024-11-14T04:56:15.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
108942024-11-14T04:56:15.531ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
108952024-11-14T04:56:15.531ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
108962024-11-14T04:56:15.531ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 214.989496ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
108972024-11-14T04:56:15.534ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
108982024-11-14T04:56:15.534ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 217.698805ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
108992024-11-14T04:56:15.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109002024-11-14T04:56:15.538ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109012024-11-14T04:56:15.539ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109022024-11-14T04:56:15.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109032024-11-14T04:56:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109042024-11-14T04:56:15.541ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109052024-11-14T04:56:15.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109062024-11-14T04:56:15.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
109072024-11-14T04:56:15.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
109082024-11-14T04:56:15.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
109092024-11-14T04:56:15.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
109102024-11-14T04:56:15.543ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
109112024-11-14T04:56:15.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
109122024-11-14T04:56:15.544ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109132024-11-14T04:56:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
109142024-11-14T04:56:15.545ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
109152024-11-14T04:56:15.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
109162024-11-14T04:56:15.546ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
109172024-11-14T04:56:15.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
109182024-11-14T04:56:15.550ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109192024-11-14T04:56:15.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109202024-11-14T04:56:15.552ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109212024-11-14T04:56:15.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109222024-11-14T04:56:15.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 241.808132ms iteration = 21 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
109232024-11-14T04:56:15.566ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
109242024-11-14T04:56:15.577ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 260.478614ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
109252024-11-14T04:56:15.580ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
109262024-11-14T04:56:15.580ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
109272024-11-14T04:56:15.580ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
109282024-11-14T04:56:15.580ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
109292024-11-14T04:56:15.581ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
109302024-11-14T04:56:15.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
109312024-11-14T04:56:15.581ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
109322024-11-14T04:56:15.581ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 264.49213ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
109332024-11-14T04:56:15.583ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109342024-11-14T04:56:15.587ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
109352024-11-14T04:56:15.587ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
109362024-11-14T04:56:15.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
109372024-11-14T04:56:15.591ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
109382024-11-14T04:56:15.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
109392024-11-14T04:56:15.600ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
109402024-11-14T04:56:15.600ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
109412024-11-14T04:56:15.600ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0bea88c-927f-407d-8ac4-9a9c4bbc4bc3", "content-length": "26", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
109422024-11-14T04:56:15.600ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
109432024-11-14T04:56:15.601ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109442024-11-14T04:56:15.602ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
109452024-11-14T04:56:15.602ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(600ec60a-2dd6-472c-b285-024765cf6c90, 0), (5f2ddd08-a8e8-4775-819d-824daf062f34, 0), (677476f2-8740-4b58-93e4-82e4fac86fd6, 0)]
109462024-11-14T04:56:15.603ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 379690369 } })
109472024-11-14T04:56:15.604ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
109482024-11-14T04:56:15.604ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
109492024-11-14T04:56:15.605ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "56fd8de3-1a2a-4ae3-9a9d-1932641246ac", "content-length": "26", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
109502024-11-14T04:56:15.605ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
109512024-11-14T04:56:15.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
109522024-11-14T04:56:15.605ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
109532024-11-14T04:56:15.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
109542024-11-14T04:56:15.608ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
109552024-11-14T04:56:15.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
109562024-11-14T04:56:15.608ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
109572024-11-14T04:56:15.609ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 292.872341ms iteration = 21 status = 0
109582024-11-14T04:56:15.609ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109592024-11-14T04:56:15.610ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109602024-11-14T04:56:15.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109612024-11-14T04:56:15.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
109622024-11-14T04:56:15.613ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
109632024-11-14T04:56:15.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
109642024-11-14T04:56:15.617ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
109652024-11-14T04:56:15.617ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
109662024-11-14T04:56:15.617ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
109672024-11-14T04:56:15.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
109682024-11-14T04:56:15.623ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 307.273849ms iteration = 23 status = {"physical_disks_added":0}
109692024-11-14T04:56:15.646ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
109702024-11-14T04:56:15.646ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
109712024-11-14T04:56:15.646ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
109722024-11-14T04:56:15.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
109732024-11-14T04:56:15.667ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
109742024-11-14T04:56:15.667ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 350.872051ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
109752024-11-14T04:56:15.721ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
109762024-11-14T04:56:15.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
109772024-11-14T04:56:15.726ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
109782024-11-14T04:56:15.726ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "d6a39713-d39d-4a66-af46-7ca63f0fc584", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
109792024-11-14T04:56:15.726ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
109802024-11-14T04:56:15.729ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "05572d2b-0b24-43cf-81b6-94778adac524", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
109812024-11-14T04:56:15.729ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
109822024-11-14T04:56:15.730ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
109832024-11-14T04:56:15.730ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 414.159754ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
109842024-11-14T04:56:15.736ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
109852024-11-14T04:56:15.736ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
109862024-11-14T04:56:15.737ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac72aeb5-50a9-489b-93c6-9bd9febf3edb", "content-length": "26", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
109872024-11-14T04:56:15.737ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
109882024-11-14T04:56:15.739ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
109892024-11-14T04:56:15.780ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
109902024-11-14T04:56:15.800ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
109912024-11-14T04:56:15.800ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f006311-4915-45cc-98a4-a1e3485e6c4a", "content-length": "1", "date": "Thu, 14 Nov 2024 04:56:14 GMT"} })
109922024-11-14T04:56:15.802ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
109932024-11-14T04:56:15.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
109942024-11-14T04:56:15.804ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
109952024-11-14T04:56:15.804ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
109962024-11-14T04:56:15.804ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "091c1d1a-5260-416c-95b3-208e537dc457", "content-length": "372", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
109972024-11-14T04:56:15.804ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
109982024-11-14T04:56:15.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
109992024-11-14T04:56:15.806ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110002024-11-14T04:56:15.806ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110012024-11-14T04:56:15.807ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110022024-11-14T04:56:15.807ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110032024-11-14T04:56:15.809ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 493.013707ms iteration = 21 status = null
110042024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
110052024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
110062024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
110072024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
110082024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
110092024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
110102024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110112024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
110122024-11-14T04:56:15.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110132024-11-14T04:56:15.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
110142024-11-14T04:56:15.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
110152024-11-14T04:56:15.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
110162024-11-14T04:56:15.818ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
110172024-11-14T04:56:15.818ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
110182024-11-14T04:56:15.818ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
110192024-11-14T04:56:15.818ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 502.309917ms iteration = 12 status = {"generation":2}
110202024-11-14T04:56:15.820ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110212024-11-14T04:56:15.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110222024-11-14T04:56:15.820ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
110232024-11-14T04:56:15.820ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
110242024-11-14T04:56:15.820ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
110252024-11-14T04:56:15.820ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 504.548251ms iteration = 12 status = {"generation":1}
110262024-11-14T04:56:15.822ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110272024-11-14T04:56:15.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110282024-11-14T04:56:15.825ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110292024-11-14T04:56:15.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110302024-11-14T04:56:15.827ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110312024-11-14T04:56:15.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110322024-11-14T04:56:15.829ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110332024-11-14T04:56:15.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110342024-11-14T04:56:15.831ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110352024-11-14T04:56:15.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110362024-11-14T04:56:15.833ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110372024-11-14T04:56:15.837ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
110382024-11-14T04:56:15.837ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110392024-11-14T04:56:15.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110402024-11-14T04:56:15.839ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110412024-11-14T04:56:15.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110422024-11-14T04:56:15.840ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110432024-11-14T04:56:15.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110442024-11-14T04:56:15.843ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110452024-11-14T04:56:15.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110462024-11-14T04:56:15.849ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110472024-11-14T04:56:15.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110482024-11-14T04:56:15.851ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110492024-11-14T04:56:15.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110502024-11-14T04:56:15.853ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
110512024-11-14T04:56:15.853ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 536.525647ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
110522024-11-14T04:56:15.853ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110532024-11-14T04:56:15.853ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110542024-11-14T04:56:15.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110552024-11-14T04:56:15.855ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110562024-11-14T04:56:15.855ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
110572024-11-14T04:56:15.855ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
110582024-11-14T04:56:15.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
110592024-11-14T04:56:15.861ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
110602024-11-14T04:56:15.861ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110612024-11-14T04:56:15.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110622024-11-14T04:56:15.863ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110632024-11-14T04:56:15.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110642024-11-14T04:56:15.864ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110652024-11-14T04:56:15.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110662024-11-14T04:56:15.866ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110672024-11-14T04:56:15.872ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
110682024-11-14T04:56:15.872ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110692024-11-14T04:56:15.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110702024-11-14T04:56:15.877ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
110712024-11-14T04:56:15.877ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
110722024-11-14T04:56:15.877ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ac92001e-eae2-4a32-a8cb-43c9dcb35fbc", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110732024-11-14T04:56:15.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
110742024-11-14T04:56:15.877ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110752024-11-14T04:56:15.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110762024-11-14T04:56:15.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
110772024-11-14T04:56:15.891ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
110782024-11-14T04:56:15.891ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
110792024-11-14T04:56:15.891ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
110802024-11-14T04:56:15.891ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfa10683-b511-4524-a263-6529b068d742", "content-length": "476", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110812024-11-14T04:56:15.892ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
110822024-11-14T04:56:15.892ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa4cb8a2-fd70-4a17-8f40-e063f06bc897", "content-length": "734", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110832024-11-14T04:56:15.892ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
110842024-11-14T04:56:15.892ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bcda95d-4b3e-4470-a3d8-457d91cd722f", "content-length": "111", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110852024-11-14T04:56:15.892ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
110862024-11-14T04:56:15.893ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91622dd9-d772-4a55-a7f0-e84ed77450e9", "content-length": "111", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110872024-11-14T04:56:15.893ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
110882024-11-14T04:56:15.893ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d50f34f-4283-43d6-b70c-dbb093c6cdf5", "content-length": "167", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110892024-11-14T04:56:15.893ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
110902024-11-14T04:56:15.893ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "81212d11-7ea8-4069-943a-81e4790b05ff", "content-length": "167", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110912024-11-14T04:56:15.893ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
110922024-11-14T04:56:15.894ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e357599a-5d09-41d3-bcfa-c40fad53adf1", "content-length": "175", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110932024-11-14T04:56:15.894ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
110942024-11-14T04:56:15.895ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 578.685638ms iteration = 21 status = {}
110952024-11-14T04:56:15.897ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b0e0ae8-0ae9-4c2d-85bf-f0fe09d11f6d", "content-length": "175", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110962024-11-14T04:56:15.897ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
110972024-11-14T04:56:15.897ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4817898f-c7a4-4825-ac42-16e0f94ef692", "content-length": "702", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
110982024-11-14T04:56:15.897ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
110992024-11-14T04:56:15.899ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "522af6c9-fce3-448f-9fda-bd93cca8a6d0", "content-length": "727", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111002024-11-14T04:56:15.899ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
111012024-11-14T04:56:15.899ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "89895498-5883-423c-a3e4-e18355e6666c", "content-length": "729", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111022024-11-14T04:56:15.899ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
111032024-11-14T04:56:15.899ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "305a4f58-6e55-4f8b-ac1f-d6a8c8f0b7c0", "content-length": "728", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111042024-11-14T04:56:15.899ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
111052024-11-14T04:56:15.899ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "905565ab-2161-40c6-bae8-7a69cd2fd408", "content-length": "722", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111062024-11-14T04:56:15.899ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
111072024-11-14T04:56:15.900ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "82b4dc67-7228-4586-aacb-f939aa24fdfc", "content-length": "109", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111082024-11-14T04:56:15.900ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
111092024-11-14T04:56:15.900ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "894a7783-a6f1-462d-a4b7-631d1eb26749", "content-length": "109", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111102024-11-14T04:56:15.900ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
111112024-11-14T04:56:15.900ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d224c723-cab0-49f3-95c1-eede31c95b94", "content-length": "169", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111122024-11-14T04:56:15.900ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
111132024-11-14T04:56:15.901ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0db1ca9e-5055-4043-a686-fc3d5cf6910a", "content-length": "169", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111142024-11-14T04:56:15.901ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
111152024-11-14T04:56:15.901ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5d06d28-aea1-4b5b-8351-6a9fc2464d22", "content-length": "178", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111162024-11-14T04:56:15.901ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
111172024-11-14T04:56:15.901ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d3238ac-68e4-4c3a-a095-8788e00cafd4", "content-length": "178", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111182024-11-14T04:56:15.901ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
111192024-11-14T04:56:15.901ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75682d5a-302b-4823-a7ce-201202bd81cd", "content-length": "702", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111202024-11-14T04:56:15.901ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
111212024-11-14T04:56:15.902ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff2fc84f-1402-4b39-85e6-6f7e92ca8f92", "content-length": "727", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111222024-11-14T04:56:15.902ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
111232024-11-14T04:56:15.902ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99660f2d-c78c-480f-9a4f-85c9512adba6", "content-length": "729", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111242024-11-14T04:56:15.902ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
111252024-11-14T04:56:15.902ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a251d187-ec04-47fc-ac8d-f94cb3f29831", "content-length": "728", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111262024-11-14T04:56:15.902ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
111272024-11-14T04:56:15.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
111282024-11-14T04:56:15.909ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
111292024-11-14T04:56:15.910ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "fa099863-35d1-4898-98ea-1660acef4864", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111302024-11-14T04:56:15.910ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111312024-11-14T04:56:15.911ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "3169a508-9380-4983-8800-8873d9777346", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111322024-11-14T04:56:15.911ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111332024-11-14T04:56:15.914ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7026d5b8-2407-46e2-88cd-9c1470299ab4", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111342024-11-14T04:56:15.914ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111352024-11-14T04:56:15.916ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111362024-11-14T04:56:15.916ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 600.637024ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
111372024-11-14T04:56:15.947ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
111382024-11-14T04:56:15.949ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111392024-11-14T04:56:15.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111402024-11-14T04:56:15.950ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111412024-11-14T04:56:15.951ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
111422024-11-14T04:56:15.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
111432024-11-14T04:56:15.952ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111442024-11-14T04:56:15.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111452024-11-14T04:56:15.953ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
111462024-11-14T04:56:15.953ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
111472024-11-14T04:56:15.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
111482024-11-14T04:56:15.954ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111492024-11-14T04:56:15.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111502024-11-14T04:56:15.955ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111512024-11-14T04:56:15.955ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
111522024-11-14T04:56:15.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
111532024-11-14T04:56:15.957ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111542024-11-14T04:56:15.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111552024-11-14T04:56:15.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
111562024-11-14T04:56:15.957ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
111572024-11-14T04:56:15.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
111582024-11-14T04:56:15.958ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
111592024-11-14T04:56:15.958ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
111602024-11-14T04:56:15.959ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111612024-11-14T04:56:15.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111622024-11-14T04:56:15.960ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111632024-11-14T04:56:15.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111642024-11-14T04:56:15.962ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111652024-11-14T04:56:15.967ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
111662024-11-14T04:56:15.967ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111672024-11-14T04:56:15.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111682024-11-14T04:56:15.968ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111692024-11-14T04:56:15.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111702024-11-14T04:56:15.970ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111712024-11-14T04:56:15.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111722024-11-14T04:56:16.005ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111732024-11-14T04:56:16.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111742024-11-14T04:56:16.020ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
111752024-11-14T04:56:16.020ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
111762024-11-14T04:56:16.022ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111772024-11-14T04:56:16.023ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
111782024-11-14T04:56:16.023ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111792024-11-14T04:56:16.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111802024-11-14T04:56:16.028ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
111812024-11-14T04:56:16.028ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111822024-11-14T04:56:16.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111832024-11-14T04:56:16.035ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 68cb12a7-f65f-4940-ad7a-12ac0e8fa220, kind: Service { id: 7d1b51fe-adf1-4350-9c25-9240054e3911 }, name: Name("nexus-7d1b51fe-adf1-4350-9c25-9240054e3911"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 137, 76])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5946ca8-a389-4a86-aed2-97d4e8639c67, kind: Service { id: 13db289d-f9b6-41e9-b3bd-5ab4f3105694 }, name: Name("nexus-13db289d-f9b6-41e9-b3bd-5ab4f3105694"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b897d57d-b436-467c-89e9-441e43e441fa, kind: Service { id: 79994e02-6747-4071-83af-7872b31640fb }, name: Name("nexus-79994e02-6747-4071-83af-7872b31640fb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 66])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 32de9250-63c9-46be-9a6b-ee4af19ab1ab, kind: Service { id: 48221b8a-d832-4d05-ae47-e92f81388c0c }, name: Name("external-dns-48221b8a-d832-4d05-ae47-e92f81388c0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 126])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: da2f10b4-11b4-4931-a7f7-d415353256ec, kind: Service { id: 8c0d54f6-76b3-47cc-9828-b8e677cc2961 }, name: Name("external-dns-8c0d54f6-76b3-47cc-9828-b8e677cc2961"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 129])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
111842024-11-14T04:56:16.035ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111852024-11-14T04:56:16.035ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
111862024-11-14T04:56:16.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111872024-11-14T04:56:16.036ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fc9e671-49ac-4db4-803e-4e46c6feadfa", "content-length": "134", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
111882024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
111892024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
111902024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
111912024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
111922024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
111932024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
111942024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
111952024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
111962024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
111972024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
111982024-11-14T04:56:16.036ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
111992024-11-14T04:56:16.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
112002024-11-14T04:56:16.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
112012024-11-14T04:56:16.037ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112022024-11-14T04:56:16.039ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
112032024-11-14T04:56:16.039ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
112042024-11-14T04:56:16.039ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112052024-11-14T04:56:16.039ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c608d5f0-7385-4020-bd1e-9683004214b7", "content-length": "2", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
112062024-11-14T04:56:16.039ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
112072024-11-14T04:56:16.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112082024-11-14T04:56:16.039ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba87a121-4082-4795-ae64-922e5c048133", "content-length": "51", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
112092024-11-14T04:56:16.039ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
112102024-11-14T04:56:16.039ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
112112024-11-14T04:56:16.040ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
112122024-11-14T04:56:16.040ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 32de9250-63c9-46be-9a6b-ee4af19ab1ab, kind: Service { id: 48221b8a-d832-4d05-ae47-e92f81388c0c }, name: Name("external-dns-48221b8a-d832-4d05-ae47-e92f81388c0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 126])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: da2f10b4-11b4-4931-a7f7-d415353256ec, kind: Service { id: 8c0d54f6-76b3-47cc-9828-b8e677cc2961 }, name: Name("external-dns-8c0d54f6-76b3-47cc-9828-b8e677cc2961"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 129])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 68cb12a7-f65f-4940-ad7a-12ac0e8fa220, kind: Service { id: 7d1b51fe-adf1-4350-9c25-9240054e3911 }, name: Name("nexus-7d1b51fe-adf1-4350-9c25-9240054e3911"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 137, 76])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5946ca8-a389-4a86-aed2-97d4e8639c67, kind: Service { id: 13db289d-f9b6-41e9-b3bd-5ab4f3105694 }, name: Name("nexus-13db289d-f9b6-41e9-b3bd-5ab4f3105694"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b897d57d-b436-467c-89e9-441e43e441fa, kind: Service { id: 79994e02-6747-4071-83af-7872b31640fb }, name: Name("nexus-79994e02-6747-4071-83af-7872b31640fb"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 66])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
112132024-11-14T04:56:16.040ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
112142024-11-14T04:56:16.070ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f34b2355-db46-4b4f-8122-1a6c560553ec", "date": "Thu, 14 Nov 2024 04:56:15 GMT"} })
112152024-11-14T04:56:16.072ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 755.604052ms iteration = 22 status = {}
112162024-11-14T04:56:16.072ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1712b6f-99a5-4351-a362-d2379b24d911", "content-length": "691", "date": "Thu, 14 Nov 2024 04:56:16 GMT"} })
112172024-11-14T04:56:16.072ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
112182024-11-14T04:56:16.072ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
112192024-11-14T04:56:16.072ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
112202024-11-14T04:56:16.074ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 757.613479ms iteration = 21 status = {}
112212024-11-14T04:56:16.153ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: c073d51c-abaa-48df-abb6-2f053e9c1bec, time_created: 2024-11-14T04:46:13.572725Z, time_modified: 2024-11-14T04:46:13.572725Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: f971eb12-0fea-4084-96d3-74198bddd289, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
112222024-11-14T04:56:16.153ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
112232024-11-14T04:56:16.153ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112242024-11-14T04:56:16.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112252024-11-14T04:56:16.155ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112262024-11-14T04:56:16.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112272024-11-14T04:56:16.157ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112282024-11-14T04:56:16.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
112292024-11-14T04:56:16.190ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
112302024-11-14T04:56:16.194ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "ec2e74b5-03a8-4659-9bdb-cc89649751d5", "date": "Thu, 14 Nov 2024 04:56:16 GMT"} })
112312024-11-14T04:56:16.196ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
112322024-11-14T04:56:16.196ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 880.022825ms
112332024-11-14T04:56:16.196ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 880.034255ms iteration = 3 status = {}
112342024-11-14T04:56:16.210ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4257014d-0912-4451-b237-fb8faa7d15d8", "date": "Thu, 14 Nov 2024 04:56:16 GMT"} })
112352024-11-14T04:56:16.210ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
112362024-11-14T04:56:16.210ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
112372024-11-14T04:56:16.211ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f914f98-7768-4d54-9a1e-40c88a977d9c", "content-length": "682", "date": "Thu, 14 Nov 2024 04:56:16 GMT"} })
112382024-11-14T04:56:16.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
112392024-11-14T04:56:16.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
112402024-11-14T04:56:16.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
112412024-11-14T04:56:16.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
112422024-11-14T04:56:16.217ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
112432024-11-14T04:56:16.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
112442024-11-14T04:56:16.219ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
112452024-11-14T04:56:16.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
112462024-11-14T04:56:16.227ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 910.738114ms iteration = 21 status = {}
112472024-11-14T04:56:16.441ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7eaee198-2f53-4ab1-8a21-94ea6c4fdb44", "content-length": "30760", "date": "Thu, 14 Nov 2024 04:56:16 GMT"} })
112482024-11-14T04:56:16.443ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
112492024-11-14T04:56:16.443ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
112502024-11-14T04:56:16.443ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
112512024-11-14T04:56:16.445ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112522024-11-14T04:56:16.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112532024-11-14T04:56:16.447ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112542024-11-14T04:56:16.449ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
112552024-11-14T04:56:16.449ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112562024-11-14T04:56:16.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112572024-11-14T04:56:16.606ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = ecb3f7af-d2d2-4097-9125-aa9cff7e1e2d file = nexus/db-queries/src/db/datastore/inventory.rs:1000
112582024-11-14T04:56:16.606ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = ecb3f7af-d2d2-4097-9125-aa9cff7e1e2d time_started = 2024-11-14 04:56:15.891536 UTC
112592024-11-14T04:56:16.606ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.289710148s iteration = 3 status = {"collection_id":"ecb3f7af-d2d2-4097-9125-aa9cff7e1e2d","time_done":"2024-11-14 04:56:16.443612 UTC","time_started":"2024-11-14 04:56:15.891536 UTC"}
112602024-11-14T04:56:16.606ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Dependency
112612024-11-14T04:56:16.606ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
112622024-11-14T04:56:16.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
112632024-11-14T04:56:16.608ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112642024-11-14T04:56:16.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
112652024-11-14T04:56:16.609ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
112662024-11-14T04:56:16.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
112672024-11-14T04:56:16.611ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
112682024-11-14T04:56:16.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
112692024-11-14T04:56:16.613ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112702024-11-14T04:56:16.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
112712024-11-14T04:56:16.615ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
112722024-11-14T04:56:16.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
112732024-11-14T04:56:16.650ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 44.667884ms iteration = 24 status = {"physical_disks_added":0}
112742024-11-14T04:56:20.242ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
112752024-11-14T04:56:20.242ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
112762024-11-14T04:56:20.272ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
112772024-11-14T04:56:20.272ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:53919
112782024-11-14T04:56:20.272ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:53919 req_id = bd9d8469-2f26-4f7f-9f83-b5af06181c6c roles = RoleSet { roles: {} } uri = /metrics/producers
112792024-11-14T04:56:20.272ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:53919 req_id = bd9d8469-2f26-4f7f-9f83-b5af06181c6c resource = Database result = Ok(()) uri = /metrics/producers
112802024-11-14T04:56:20.273ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 974 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = c6327811-140d-41a0-ae58-5685da2f2378 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
112812024-11-14T04:56:20.312ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
112822024-11-14T04:56:20.312ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39773 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:53919 req_id = bd9d8469-2f26-4f7f-9f83-b5af06181c6c response_code = 201 uri = /metrics/producers
112832024-11-14T04:56:20.312ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "bd9d8469-2f26-4f7f-9f83-b5af06181c6c", "content-length": "41", "date": "Thu, 14 Nov 2024 04:56:20 GMT"} })
112842024-11-14T04:56:20.314ZDEBGnexus (producer-registration-task): registered with nexus successfully
112852024-11-14T04:56:20.314ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
112862024-11-14T04:56:25.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
112872024-11-14T04:56:25.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
112882024-11-14T04:56:25.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112892024-11-14T04:56:25.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112902024-11-14T04:56:25.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
112912024-11-14T04:56:25.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112922024-11-14T04:56:25.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112932024-11-14T04:56:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
112942024-11-14T04:56:25.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112952024-11-14T04:56:25.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112962024-11-14T04:56:25.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112972024-11-14T04:56:25.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112982024-11-14T04:56:25.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112992024-11-14T04:56:25.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
113002024-11-14T04:56:25.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113012024-11-14T04:56:25.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113022024-11-14T04:56:25.337ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
113032024-11-14T04:56:25.337ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.40388ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
113042024-11-14T04:56:25.348ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113052024-11-14T04:56:25.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113062024-11-14T04:56:25.376ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 61.112569ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
113072024-11-14T04:56:35.257ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1340 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 8ece195f-cce1-4751-b764-edca700c4bb6 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
113082024-11-14T04:56:35.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
113092024-11-14T04:56:35.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113102024-11-14T04:56:35.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
113112024-11-14T04:56:35.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113122024-11-14T04:56:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113132024-11-14T04:56:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113142024-11-14T04:56:35.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113152024-11-14T04:56:35.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113162024-11-14T04:56:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113172024-11-14T04:56:35.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.279034ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
113182024-11-14T04:56:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
113192024-11-14T04:56:35.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113202024-11-14T04:56:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113212024-11-14T04:56:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113222024-11-14T04:56:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113232024-11-14T04:56:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113242024-11-14T04:56:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
113252024-11-14T04:56:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113262024-11-14T04:56:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113272024-11-14T04:56:35.338ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
113282024-11-14T04:56:35.338ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.743249ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
113292024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
113302024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
113312024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113322024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
113332024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
113342024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
113352024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
113362024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
113372024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
113382024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
113392024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
113402024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
113412024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
113422024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
113432024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
113442024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
113452024-11-14T04:56:45.316ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
113462024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
113472024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
113482024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
113492024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113502024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113512024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
113522024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
113532024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
113542024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
113552024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113562024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
113572024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
113582024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
113592024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
113602024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
113612024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
113622024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
113632024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
113642024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
113652024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
113662024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
113672024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
113682024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113692024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
113702024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113712024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
113722024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
113732024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
113742024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113752024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
113762024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113772024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
113782024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
113792024-11-14T04:56:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
113802024-11-14T04:56:45.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113812024-11-14T04:56:45.319ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.723278ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
113822024-11-14T04:56:45.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113832024-11-14T04:56:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
113842024-11-14T04:56:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
113852024-11-14T04:56:45.320ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
113862024-11-14T04:56:45.320ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
113872024-11-14T04:56:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
113882024-11-14T04:56:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
113892024-11-14T04:56:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
113902024-11-14T04:56:45.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113912024-11-14T04:56:45.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
113922024-11-14T04:56:45.322ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113932024-11-14T04:56:45.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113942024-11-14T04:56:45.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113952024-11-14T04:56:45.324ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 100876660 } })
113962024-11-14T04:56:45.324ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113972024-11-14T04:56:45.324ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
113982024-11-14T04:56:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
113992024-11-14T04:56:45.325ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114002024-11-14T04:56:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114012024-11-14T04:56:45.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114022024-11-14T04:56:45.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
114032024-11-14T04:56:45.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114042024-11-14T04:56:45.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
114052024-11-14T04:56:45.327ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114062024-11-14T04:56:45.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114072024-11-14T04:56:45.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114082024-11-14T04:56:45.329ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
114092024-11-14T04:56:45.329ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114102024-11-14T04:56:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114112024-11-14T04:56:45.330ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
114122024-11-14T04:56:45.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
114132024-11-14T04:56:45.333ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
114142024-11-14T04:56:45.333ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
114152024-11-14T04:56:45.333ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
114162024-11-14T04:56:45.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
114172024-11-14T04:56:45.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114182024-11-14T04:56:45.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114192024-11-14T04:56:45.336ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114202024-11-14T04:56:45.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114212024-11-14T04:56:45.338ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 21.75557ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
114222024-11-14T04:56:45.338ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 114853355 } })
114232024-11-14T04:56:45.338ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114242024-11-14T04:56:45.338ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 115137753 } })
114252024-11-14T04:56:45.338ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
114262024-11-14T04:56:45.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114272024-11-14T04:56:45.338ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
114282024-11-14T04:56:45.371ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 55.210969ms iteration = 22 status = {"errors":[],"records_set_to_done":[]}
114292024-11-14T04:56:45.378ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
114302024-11-14T04:56:45.378ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
114312024-11-14T04:56:45.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
114322024-11-14T04:56:45.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
114332024-11-14T04:56:45.380ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 64.286472ms iteration = 22 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
114342024-11-14T04:56:45.384ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114352024-11-14T04:56:45.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114362024-11-14T04:56:45.387ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114372024-11-14T04:56:45.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114382024-11-14T04:56:45.389ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114392024-11-14T04:56:45.390ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
114402024-11-14T04:56:45.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
114412024-11-14T04:56:45.392ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
114422024-11-14T04:56:45.392ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
114432024-11-14T04:56:45.392ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 75.916878ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
114442024-11-14T04:56:45.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
114452024-11-14T04:56:45.392ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114462024-11-14T04:56:45.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114472024-11-14T04:56:45.393ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.902217ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
114482024-11-14T04:56:45.393ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114492024-11-14T04:56:45.394ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114502024-11-14T04:56:45.394ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 77.970658ms iteration = 22 status = 0
114512024-11-14T04:56:45.394ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
114522024-11-14T04:56:45.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
114532024-11-14T04:56:45.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
114542024-11-14T04:56:45.395ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114552024-11-14T04:56:45.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114562024-11-14T04:56:45.396ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
114572024-11-14T04:56:45.396ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114582024-11-14T04:56:45.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114592024-11-14T04:56:45.398ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
114602024-11-14T04:56:45.398ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 82.059966ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
114612024-11-14T04:56:45.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114622024-11-14T04:56:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
114632024-11-14T04:56:45.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114642024-11-14T04:56:45.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114652024-11-14T04:56:45.423ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
114662024-11-14T04:56:45.423ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
114672024-11-14T04:56:45.423ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "31960fd7-918e-4b44-8cc9-c6a81fa18f26", "content-length": "26", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
114682024-11-14T04:56:45.423ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
114692024-11-14T04:56:45.425ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
114702024-11-14T04:56:45.429ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 113.800136ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
114712024-11-14T04:56:45.436ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
114722024-11-14T04:56:45.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
114732024-11-14T04:56:45.448ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
114742024-11-14T04:56:45.448ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
114752024-11-14T04:56:45.449ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3493aca-ab9f-4cee-915c-3d5a3086f373", "content-length": "26", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
114762024-11-14T04:56:45.449ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
114772024-11-14T04:56:45.451ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
114782024-11-14T04:56:45.451ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
114792024-11-14T04:56:45.451ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
114802024-11-14T04:56:45.452ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e871bb2-1946-4ca9-850a-b03d1575d163", "content-length": "26", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
114812024-11-14T04:56:45.452ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
114822024-11-14T04:56:45.454ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
114832024-11-14T04:56:45.509ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 193.673069ms iteration = 25 status = {"physical_disks_added":0}
114842024-11-14T04:56:45.518ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
114852024-11-14T04:56:45.518ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
114862024-11-14T04:56:45.519ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "65e61b2a-9848-4c2b-91b2-891a204bf9dd", "content-length": "372", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
114872024-11-14T04:56:45.519ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114882024-11-14T04:56:45.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114892024-11-14T04:56:45.529ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
114902024-11-14T04:56:45.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
114912024-11-14T04:56:45.529ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
114922024-11-14T04:56:45.530ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b79d2b96-d945-4150-8cf5-933b9d1bf295", "content-length": "1", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
114932024-11-14T04:56:45.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114942024-11-14T04:56:45.531ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
114952024-11-14T04:56:45.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
114962024-11-14T04:56:45.533ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
114972024-11-14T04:56:45.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
114982024-11-14T04:56:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
114992024-11-14T04:56:45.533ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 217.648463ms iteration = 22 status = null
115002024-11-14T04:56:45.533ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
115012024-11-14T04:56:45.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 217.975222ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
115022024-11-14T04:56:45.558ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 241.828833ms iteration = 22 status = {}
115032024-11-14T04:56:45.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 242.349399ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
115042024-11-14T04:56:45.565ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
115052024-11-14T04:56:45.565ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
115062024-11-14T04:56:45.565ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8aac57f2-cfcc-4793-b2ff-a76d5ed8643d", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
115072024-11-14T04:56:45.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115082024-11-14T04:56:45.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115092024-11-14T04:56:45.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115102024-11-14T04:56:45.568ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115112024-11-14T04:56:45.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115122024-11-14T04:56:45.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
115132024-11-14T04:56:45.570ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115142024-11-14T04:56:45.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
115152024-11-14T04:56:45.571ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115162024-11-14T04:56:45.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115172024-11-14T04:56:45.580ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
115182024-11-14T04:56:45.581ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
115192024-11-14T04:56:45.581ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8532d37a-199e-442a-a737-7308aadfeec8", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
115202024-11-14T04:56:45.583ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 267.513407ms iteration = 23 status = {}
115212024-11-14T04:56:45.629ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115222024-11-14T04:56:45.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115232024-11-14T04:56:45.632ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
115242024-11-14T04:56:45.632ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115252024-11-14T04:56:45.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115262024-11-14T04:56:45.658ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
115272024-11-14T04:56:45.658ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "338194fb-8c74-4a44-a004-dc41f9afd755", "content-length": "691", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
115282024-11-14T04:56:45.658ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
115292024-11-14T04:56:45.658ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
115302024-11-14T04:56:45.658ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
115312024-11-14T04:56:45.660ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 343.94685ms iteration = 22 status = {}
115322024-11-14T04:56:45.668ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
115332024-11-14T04:56:45.669ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "45bfccea-159d-4d40-bc12-47007a0659d5", "content-length": "134", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
115342024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
115352024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
115362024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
115372024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
115382024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115392024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115402024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115412024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
115422024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115432024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
115442024-11-14T04:56:45.669ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
115452024-11-14T04:56:45.669ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115462024-11-14T04:56:45.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115472024-11-14T04:56:45.671ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
115482024-11-14T04:56:45.672ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f95847d-40c7-4025-8417-7b56e90cddaa", "content-length": "2", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
115492024-11-14T04:56:45.672ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
115502024-11-14T04:56:45.672ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "d04b1648-fcbc-40be-877e-35f291dfe334", "content-length": "51", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
115512024-11-14T04:56:45.672ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
115522024-11-14T04:56:45.672ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
115532024-11-14T04:56:45.773ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8027b713-f820-40c4-87ae-ea67f2ed0666", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
115542024-11-14T04:56:45.773ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
115552024-11-14T04:56:45.773ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
115562024-11-14T04:56:45.777ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e1d6fa6-7047-4b50-979e-e42fd18bd90b", "content-length": "682", "date": "Thu, 14 Nov 2024 04:56:45 GMT"} })
115572024-11-14T04:56:45.777ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115582024-11-14T04:56:45.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115592024-11-14T04:56:45.780ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115602024-11-14T04:56:45.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115612024-11-14T04:56:45.783ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115622024-11-14T04:56:45.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115632024-11-14T04:56:45.785ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115642024-11-14T04:56:45.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115652024-11-14T04:56:45.792ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 476.2104ms iteration = 22 status = {}
115662024-11-14T04:56:49.855ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:44218
115672024-11-14T04:56:49.855ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:44218 req_id = 256c96cc-c706-4652-8ebc-1bce6bf5205e roles = RoleSet { roles: {} } uri = /metrics/producers
115682024-11-14T04:56:49.855ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:44218 req_id = 256c96cc-c706-4652-8ebc-1bce6bf5205e resource = Database result = Ok(()) uri = /metrics/producers
115692024-11-14T04:56:49.892ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = e5586e29-e301-4b24-9a2c-39545da75716
115702024-11-14T04:56:49.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37051 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:44218 req_id = 256c96cc-c706-4652-8ebc-1bce6bf5205e response_code = 201 uri = /metrics/producers
115712024-11-14T04:56:50.260ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1928 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = cb549f8a-bf51-4933-b533-d0c95e51086f response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
115722024-11-14T04:56:55.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 05e6ad74-13ca-424b-9b20-b47bbb42ea06 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
115732024-11-14T04:56:55.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 05e6ad74-13ca-424b-9b20-b47bbb42ea06 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
115742024-11-14T04:56:55.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3008 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 05e6ad74-13ca-424b-9b20-b47bbb42ea06 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
115752024-11-14T04:56:55.253ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 2add788a-ca75-485e-8238-e588fbda5695 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
115762024-11-14T04:56:55.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 2add788a-ca75-485e-8238-e588fbda5695 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
115772024-11-14T04:56:55.256ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2797 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 2add788a-ca75-485e-8238-e588fbda5695 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
115782024-11-14T04:56:55.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
115792024-11-14T04:56:55.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
115802024-11-14T04:56:55.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
115812024-11-14T04:56:55.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115822024-11-14T04:56:55.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
115832024-11-14T04:56:55.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115842024-11-14T04:56:55.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115852024-11-14T04:56:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
115862024-11-14T04:56:55.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115872024-11-14T04:56:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115882024-11-14T04:56:55.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115892024-11-14T04:56:55.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115902024-11-14T04:56:55.347ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
115912024-11-14T04:56:55.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
115922024-11-14T04:56:55.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115932024-11-14T04:56:55.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
115942024-11-14T04:56:55.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115952024-11-14T04:56:55.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115962024-11-14T04:56:55.367ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
115972024-11-14T04:56:55.367ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.49349ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
115982024-11-14T04:56:55.376ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 61.402769ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
115992024-11-14T04:57:00.887ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:51842
116002024-11-14T04:57:00.887ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:51842 req_id = 59041c08-4b18-42f8-8fb2-3a9b390c9709 roles = RoleSet { roles: {} } uri = /metrics/producers
116012024-11-14T04:57:00.888ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:51842 req_id = 59041c08-4b18-42f8-8fb2-3a9b390c9709 resource = Database result = Ok(()) uri = /metrics/producers
116022024-11-14T04:57:00.924ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = f4c8f330-91d6-4976-b637-c72f900d8c87
116032024-11-14T04:57:00.924ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36478 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:51842 req_id = 59041c08-4b18-42f8-8fb2-3a9b390c9709 response_code = 201 uri = /metrics/producers
116042024-11-14T04:57:02.741ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:40571
116052024-11-14T04:57:02.741ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40571 req_id = 3b4a1494-060a-42e2-b355-1d8ec630bc74 roles = RoleSet { roles: {} } uri = /metrics/producers
116062024-11-14T04:57:02.742ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40571 req_id = 3b4a1494-060a-42e2-b355-1d8ec630bc74 resource = Database result = Ok(()) uri = /metrics/producers
116072024-11-14T04:57:02.752ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 18d4ee8d-1d67-44aa-96f9-89dfcadba34d
116082024-11-14T04:57:02.752ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10624 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40571 req_id = 3b4a1494-060a-42e2-b355-1d8ec630bc74 response_code = 201 uri = /metrics/producers
116092024-11-14T04:57:05.258ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1235 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 00dfecff-c346-4c80-80ac-2ac1f02ee0a2 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
116102024-11-14T04:57:05.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
116112024-11-14T04:57:05.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116122024-11-14T04:57:05.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
116132024-11-14T04:57:05.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116142024-11-14T04:57:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116152024-11-14T04:57:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116162024-11-14T04:57:05.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116172024-11-14T04:57:05.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116182024-11-14T04:57:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116192024-11-14T04:57:05.318ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.867532ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
116202024-11-14T04:57:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
116212024-11-14T04:57:05.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116222024-11-14T04:57:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116232024-11-14T04:57:05.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116242024-11-14T04:57:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116252024-11-14T04:57:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116262024-11-14T04:57:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
116272024-11-14T04:57:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116282024-11-14T04:57:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116292024-11-14T04:57:05.336ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
116302024-11-14T04:57:05.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.816892ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
116312024-11-14T04:57:14.539ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:34022
116322024-11-14T04:57:14.539ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:34022 req_id = 97b4efc4-3cfa-44af-8184-a56e4f40df5e roles = RoleSet { roles: {} } uri = /metrics/producers
116332024-11-14T04:57:14.539ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:34022 req_id = 97b4efc4-3cfa-44af-8184-a56e4f40df5e resource = Database result = Ok(()) uri = /metrics/producers
116342024-11-14T04:57:14.574ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = bc37b1eb-3677-438e-a276-b6d3e6fe3f8e
116352024-11-14T04:57:14.574ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35370 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:34022 req_id = 97b4efc4-3cfa-44af-8184-a56e4f40df5e response_code = 201 uri = /metrics/producers
116362024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
116372024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
116382024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
116392024-11-14T04:57:15.315ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
116402024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116412024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
116422024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
116432024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
116442024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
116452024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
116462024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
116472024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
116482024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
116492024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
116502024-11-14T04:57:15.315ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
116512024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.481\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
116522024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
116532024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
116542024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
116552024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
116562024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
116572024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
116582024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
116592024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
116602024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
116612024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
116622024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
116632024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
116642024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
116652024-11-14T04:57:15.315ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:47:15.315548846 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
116662024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
116672024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
116682024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
116692024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
116702024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
116712024-11-14T04:57:15.315ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
116722024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
116732024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
116742024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
116752024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116762024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
116772024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
116782024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
116792024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
116802024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116812024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
116822024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
116832024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
116842024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116852024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
116862024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
116872024-11-14T04:57:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116882024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
116892024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
116902024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
116912024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
116922024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
116932024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
116942024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
116952024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
116962024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
116972024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
116982024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
116992024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
117002024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
117012024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
117022024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
117032024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
117042024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
117052024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
117062024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117072024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
117082024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
117092024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
117102024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
117112024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
117122024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117132024-11-14T04:57:15.316ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 93068429 } })
117142024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
117152024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
117162024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
117172024-11-14T04:57:15.316ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 93128321 } })
117182024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
117192024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
117202024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
117212024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
117222024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
117232024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
117242024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
117252024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.097352ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
117262024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
117272024-11-14T04:57:15.316ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.292927ms iteration = 13 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
117282024-11-14T04:57:15.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117292024-11-14T04:57:15.318ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117302024-11-14T04:57:15.318ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 95533281 } })
117312024-11-14T04:57:15.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117322024-11-14T04:57:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
117332024-11-14T04:57:15.319ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
117342024-11-14T04:57:15.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117352024-11-14T04:57:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.268184ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
117362024-11-14T04:57:15.320ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
117372024-11-14T04:57:15.320ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
117382024-11-14T04:57:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
117392024-11-14T04:57:15.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
117402024-11-14T04:57:15.321ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117412024-11-14T04:57:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
117422024-11-14T04:57:15.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
117432024-11-14T04:57:15.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
117442024-11-14T04:57:15.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
117452024-11-14T04:57:15.328ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 104916202 } })
117462024-11-14T04:57:15.328ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
117472024-11-14T04:57:15.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117482024-11-14T04:57:15.329ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:47:15.315548846 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
117492024-11-14T04:57:15.329ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.282694ms iteration = 12 status = {"expiration":"2024-11-14T04:47:15.315548846Z","pruned":[]}
117502024-11-14T04:57:15.330ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117512024-11-14T04:57:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
117522024-11-14T04:57:15.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117532024-11-14T04:57:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
117542024-11-14T04:57:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
117552024-11-14T04:57:15.334ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117562024-11-14T04:57:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117572024-11-14T04:57:15.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
117582024-11-14T04:57:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
117592024-11-14T04:57:15.338ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117602024-11-14T04:57:15.339ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117612024-11-14T04:57:15.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
117622024-11-14T04:57:15.342ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
117632024-11-14T04:57:15.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
117642024-11-14T04:57:15.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
117652024-11-14T04:57:15.344ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
117662024-11-14T04:57:15.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
117672024-11-14T04:57:15.348ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 32.605474ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
117682024-11-14T04:57:15.357ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117692024-11-14T04:57:15.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
117702024-11-14T04:57:15.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117712024-11-14T04:57:15.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
117722024-11-14T04:57:15.373ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117732024-11-14T04:57:15.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117742024-11-14T04:57:15.410ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
117752024-11-14T04:57:15.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
117762024-11-14T04:57:15.412ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 96.742581ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
117772024-11-14T04:57:15.412ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
117782024-11-14T04:57:15.412ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
117792024-11-14T04:57:15.412ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
117802024-11-14T04:57:15.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
117812024-11-14T04:57:15.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117822024-11-14T04:57:15.417ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 193946800 } })
117832024-11-14T04:57:15.417ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
117842024-11-14T04:57:15.417ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
117852024-11-14T04:57:15.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117862024-11-14T04:57:15.421ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
117872024-11-14T04:57:15.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117882024-11-14T04:57:15.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
117892024-11-14T04:57:15.425ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
117902024-11-14T04:57:15.425ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
117912024-11-14T04:57:15.425ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 109.447979ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
117922024-11-14T04:57:15.425ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117932024-11-14T04:57:15.426ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117942024-11-14T04:57:15.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
117952024-11-14T04:57:15.430ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
117962024-11-14T04:57:15.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
117972024-11-14T04:57:15.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
117982024-11-14T04:57:15.430ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117992024-11-14T04:57:15.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
118002024-11-14T04:57:15.431ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
118012024-11-14T04:57:15.431ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
118022024-11-14T04:57:15.431ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
118032024-11-14T04:57:15.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
118042024-11-14T04:57:15.432ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 117.353439ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
118052024-11-14T04:57:15.434ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 118.624326ms iteration = 23 status = {"errors":[],"records_set_to_done":[]}
118062024-11-14T04:57:15.440ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
118072024-11-14T04:57:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
118082024-11-14T04:57:15.444ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118092024-11-14T04:57:15.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
118102024-11-14T04:57:15.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
118112024-11-14T04:57:15.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
118122024-11-14T04:57:15.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
118132024-11-14T04:57:15.447ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
118142024-11-14T04:57:15.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118152024-11-14T04:57:15.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118162024-11-14T04:57:15.448ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118172024-11-14T04:57:15.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118182024-11-14T04:57:15.449ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 133.584269ms iteration = 23 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
118192024-11-14T04:57:15.450ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
118202024-11-14T04:57:15.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
118212024-11-14T04:57:15.451ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
118222024-11-14T04:57:15.451ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 136.172783ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
118232024-11-14T04:57:15.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
118242024-11-14T04:57:15.452ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
118252024-11-14T04:57:15.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
118262024-11-14T04:57:15.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
118272024-11-14T04:57:15.456ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
118282024-11-14T04:57:15.456ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
118292024-11-14T04:57:15.456ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 140.839629ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
118302024-11-14T04:57:15.456ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
118312024-11-14T04:57:15.456ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 140.939741ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
118322024-11-14T04:57:15.458ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118332024-11-14T04:57:15.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
118342024-11-14T04:57:15.466ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
118352024-11-14T04:57:15.470ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
118362024-11-14T04:57:15.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
118372024-11-14T04:57:15.473ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
118382024-11-14T04:57:15.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
118392024-11-14T04:57:15.477ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
118402024-11-14T04:57:15.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
118412024-11-14T04:57:15.482ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
118422024-11-14T04:57:15.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
118432024-11-14T04:57:15.484ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118442024-11-14T04:57:15.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
118452024-11-14T04:57:15.488ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
118462024-11-14T04:57:15.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
118472024-11-14T04:57:15.498ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 182.657409ms iteration = 23 status = 0
118482024-11-14T04:57:15.506ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
118492024-11-14T04:57:15.506ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
118502024-11-14T04:57:15.506ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
118512024-11-14T04:57:15.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
118522024-11-14T04:57:15.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118532024-11-14T04:57:15.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
118542024-11-14T04:57:15.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
118552024-11-14T04:57:15.508ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 192.389662ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
118562024-11-14T04:57:15.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118572024-11-14T04:57:15.509ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
118582024-11-14T04:57:15.509ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
118592024-11-14T04:57:15.509ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 193.562235ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
118602024-11-14T04:57:15.509ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118612024-11-14T04:57:15.511ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
118622024-11-14T04:57:15.511ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118632024-11-14T04:57:15.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
118642024-11-14T04:57:15.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
118652024-11-14T04:57:15.513ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
118662024-11-14T04:57:15.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
118672024-11-14T04:57:15.514ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 198.345613ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
118682024-11-14T04:57:15.523ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
118692024-11-14T04:57:15.523ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
118702024-11-14T04:57:15.523ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
118712024-11-14T04:57:15.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118722024-11-14T04:57:15.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118732024-11-14T04:57:15.524ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
118742024-11-14T04:57:15.524ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
118752024-11-14T04:57:15.525ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "44e0cc0b-8404-4ea4-a4f5-a4569ced3db2", "content-length": "26", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
118762024-11-14T04:57:15.525ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
118772024-11-14T04:57:15.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118782024-11-14T04:57:15.527ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
118792024-11-14T04:57:15.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
118802024-11-14T04:57:15.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118812024-11-14T04:57:15.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118822024-11-14T04:57:15.532ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
118832024-11-14T04:57:15.535ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
118842024-11-14T04:57:15.535ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
118852024-11-14T04:57:15.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118862024-11-14T04:57:15.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118872024-11-14T04:57:15.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118882024-11-14T04:57:15.542ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
118892024-11-14T04:57:15.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
118902024-11-14T04:57:15.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
118912024-11-14T04:57:15.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118922024-11-14T04:57:15.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118932024-11-14T04:57:15.544ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118942024-11-14T04:57:15.547ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
118952024-11-14T04:57:15.547ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
118962024-11-14T04:57:15.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
118972024-11-14T04:57:15.548ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
118982024-11-14T04:57:15.548ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
118992024-11-14T04:57:15.549ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
119002024-11-14T04:57:15.549ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 233.664087ms iteration = 13 status = {"generation":1}
119012024-11-14T04:57:15.549ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119022024-11-14T04:57:15.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
119032024-11-14T04:57:15.552ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
119042024-11-14T04:57:15.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
119052024-11-14T04:57:15.560ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
119062024-11-14T04:57:15.560ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
119072024-11-14T04:57:15.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
119082024-11-14T04:57:15.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
119092024-11-14T04:57:15.561ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119102024-11-14T04:57:15.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
119112024-11-14T04:57:15.566ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
119122024-11-14T04:57:15.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
119132024-11-14T04:57:15.567ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
119142024-11-14T04:57:15.567ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 251.905635ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
119152024-11-14T04:57:15.568ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
119162024-11-14T04:57:15.568ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
119172024-11-14T04:57:15.568ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
119182024-11-14T04:57:15.568ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 252.957466ms iteration = 13 status = {"generation":2}
119192024-11-14T04:57:15.601ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
119202024-11-14T04:57:15.601ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
119212024-11-14T04:57:15.601ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "18a64f44-f120-488f-8bdc-fdbb1afaf0d6", "content-length": "26", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
119222024-11-14T04:57:15.601ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
119232024-11-14T04:57:15.603ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
119242024-11-14T04:57:15.604ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
119252024-11-14T04:57:15.604ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 288.600387ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
119262024-11-14T04:57:15.619ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 303.597402ms iteration = 26 status = {"physical_disks_added":0}
119272024-11-14T04:57:15.637ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
119282024-11-14T04:57:15.637ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
119292024-11-14T04:57:15.640ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "21f4581f-1a5d-4da8-88da-1973e78a365f", "content-length": "26", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
119302024-11-14T04:57:15.640ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
119312024-11-14T04:57:15.642ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
119322024-11-14T04:57:15.651ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
119332024-11-14T04:57:15.716ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
119342024-11-14T04:57:15.717ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "6a384ff8-4a7d-401c-96a8-fb0b8faec6e1", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
119352024-11-14T04:57:15.717ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
119362024-11-14T04:57:15.720ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "42f7997b-f612-4528-a8cc-64eb4faf9e90", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
119372024-11-14T04:57:15.720ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
119382024-11-14T04:57:15.722ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
119392024-11-14T04:57:15.722ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 407.392557ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
119402024-11-14T04:57:15.723ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
119412024-11-14T04:57:15.723ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8651af9-4dba-43d7-b21b-94ee0e565099", "content-length": "1", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
119422024-11-14T04:57:15.725ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
119432024-11-14T04:57:15.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
119442024-11-14T04:57:15.728ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 413.406321ms iteration = 23 status = null
119452024-11-14T04:57:15.745ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
119462024-11-14T04:57:15.745ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
119472024-11-14T04:57:15.746ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "df893fdf-312f-45a4-ad18-ce551ae6e3c3", "content-length": "372", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
119482024-11-14T04:57:15.746ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
119492024-11-14T04:57:15.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
119502024-11-14T04:57:15.757ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
119512024-11-14T04:57:15.757ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "96397c79-bc96-49ef-a84c-6f980394096a", "content-length": "691", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
119522024-11-14T04:57:15.757ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
119532024-11-14T04:57:15.757ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
119542024-11-14T04:57:15.757ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
119552024-11-14T04:57:15.759ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 444.541124ms iteration = 23 status = {}
119562024-11-14T04:57:15.785ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 470.035782ms iteration = 23 status = {}
119572024-11-14T04:57:15.796ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
119582024-11-14T04:57:15.797ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "0bd7db5a-8b09-47e0-827b-2dbb6fdcecd0", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
119592024-11-14T04:57:15.797ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
119602024-11-14T04:57:15.799ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7868fb90-b4df-4021-90af-9d2a64bc9e06", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
119612024-11-14T04:57:15.799ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
119622024-11-14T04:57:15.801ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "11b28c4e-3d4c-49f9-b02e-58b278c46003", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
119632024-11-14T04:57:15.801ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
119642024-11-14T04:57:15.803ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
119652024-11-14T04:57:15.803ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 487.96275ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
119662024-11-14T04:57:15.804ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
119672024-11-14T04:57:15.804ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
119682024-11-14T04:57:15.804ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ec9dbcec-03e8-4f72-9b5d-85796521218c", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
119692024-11-14T04:57:15.805ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
119702024-11-14T04:57:15.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
119712024-11-14T04:57:15.808ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
119722024-11-14T04:57:15.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
119732024-11-14T04:57:15.812ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
119742024-11-14T04:57:15.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
119752024-11-14T04:57:15.816ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
119762024-11-14T04:57:15.816ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
119772024-11-14T04:57:15.816ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8249c507-7350-44dd-a52d-f9c76466e973", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
119782024-11-14T04:57:15.818ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 503.352096ms iteration = 24 status = {}
119792024-11-14T04:57:15.865ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119802024-11-14T04:57:15.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119812024-11-14T04:57:15.868ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
119822024-11-14T04:57:15.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119832024-11-14T04:57:15.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119842024-11-14T04:57:15.881ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
119852024-11-14T04:57:15.882ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e3bef56-2da4-46ee-bc9c-d13e5211a113", "content-length": "134", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
119862024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
119872024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
119882024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
119892024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
119902024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
119912024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
119922024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
119932024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
119942024-11-14T04:57:15.882ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
119952024-11-14T04:57:15.883ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
119962024-11-14T04:57:15.883ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
119972024-11-14T04:57:15.883ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119982024-11-14T04:57:15.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119992024-11-14T04:57:15.885ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
120002024-11-14T04:57:15.886ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c4cd337-fc34-4e19-a165-c05071ae9b5f", "content-length": "2", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
120012024-11-14T04:57:15.886ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
120022024-11-14T04:57:15.886ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3ab6e4b-6d06-489b-9294-6f40a39ba439", "content-length": "51", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
120032024-11-14T04:57:15.886ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
120042024-11-14T04:57:15.886ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
120052024-11-14T04:57:16.006ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e21b1209-fa36-4b7e-9fa8-57aafa26fca1", "date": "Thu, 14 Nov 2024 04:57:15 GMT"} })
120062024-11-14T04:57:16.006ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
120072024-11-14T04:57:16.006ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
120082024-11-14T04:57:16.010ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "639a428f-d98f-4b1d-8a35-7e7011053a7d", "content-length": "682", "date": "Thu, 14 Nov 2024 04:57:16 GMT"} })
120092024-11-14T04:57:16.010ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120102024-11-14T04:57:16.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120112024-11-14T04:57:16.014ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120122024-11-14T04:57:16.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120132024-11-14T04:57:16.018ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120142024-11-14T04:57:16.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120152024-11-14T04:57:16.020ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120162024-11-14T04:57:16.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120172024-11-14T04:57:16.027ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 711.74358ms iteration = 23 status = {}
120182024-11-14T04:57:19.270ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:42889
120192024-11-14T04:57:19.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:42889 req_id = 650b60c4-aff6-4772-8a62-1d9837d25ae1 roles = RoleSet { roles: {} } uri = /metrics/producers
120202024-11-14T04:57:19.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:42889 req_id = 650b60c4-aff6-4772-8a62-1d9837d25ae1 resource = Database result = Ok(()) uri = /metrics/producers
120212024-11-14T04:57:19.309ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = c073d51c-abaa-48df-abb6-2f053e9c1bec
120222024-11-14T04:57:19.309ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38784 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:42889 req_id = 650b60c4-aff6-4772-8a62-1d9837d25ae1 response_code = 201 uri = /metrics/producers
120232024-11-14T04:57:20.256ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1162 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 2f53d677-f1a8-4f59-a03a-0475499bafdf response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
120242024-11-14T04:57:25.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
120252024-11-14T04:57:25.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
120262024-11-14T04:57:25.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120272024-11-14T04:57:25.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120282024-11-14T04:57:25.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120292024-11-14T04:57:25.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120302024-11-14T04:57:25.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120312024-11-14T04:57:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
120322024-11-14T04:57:25.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120332024-11-14T04:57:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120342024-11-14T04:57:25.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120352024-11-14T04:57:25.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120362024-11-14T04:57:25.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120372024-11-14T04:57:25.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
120382024-11-14T04:57:25.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120392024-11-14T04:57:25.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120402024-11-14T04:57:25.336ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
120412024-11-14T04:57:25.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.461642ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
120422024-11-14T04:57:25.344ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120432024-11-14T04:57:25.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120442024-11-14T04:57:25.372ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.890107ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
120452024-11-14T04:57:35.256ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1048 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 40c658eb-f7f5-405d-b959-9ce705f078b8 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
120462024-11-14T04:57:35.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
120472024-11-14T04:57:35.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
120482024-11-14T04:57:35.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120492024-11-14T04:57:35.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120502024-11-14T04:57:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120512024-11-14T04:57:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120522024-11-14T04:57:35.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120532024-11-14T04:57:35.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120542024-11-14T04:57:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120552024-11-14T04:57:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
120562024-11-14T04:57:35.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120572024-11-14T04:57:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120582024-11-14T04:57:35.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120592024-11-14T04:57:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120602024-11-14T04:57:35.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120612024-11-14T04:57:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
120622024-11-14T04:57:35.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120632024-11-14T04:57:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120642024-11-14T04:57:35.336ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
120652024-11-14T04:57:35.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.606647ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
120662024-11-14T04:57:35.345ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.489894ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
120672024-11-14T04:57:41.762ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:41512
120682024-11-14T04:57:41.762ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:41512 req_id = 26621171-2a00-4bb2-b48f-0b975c62fd1d roles = RoleSet { roles: {} } uri = /metrics/producers
120692024-11-14T04:57:41.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:41512 req_id = 26621171-2a00-4bb2-b48f-0b975c62fd1d resource = Database result = Ok(()) uri = /metrics/producers
120702024-11-14T04:57:41.802ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = b584ea61-acb3-47a3-a173-0d03f6cae9c9
120712024-11-14T04:57:41.802ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39466 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:41512 req_id = 26621171-2a00-4bb2-b48f-0b975c62fd1d response_code = 201 uri = /metrics/producers
120722024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
120732024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
120742024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120752024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
120762024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
120772024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
120782024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
120792024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
120802024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
120812024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
120822024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
120832024-11-14T04:57:45.314ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
120842024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
120852024-11-14T04:57:45.314ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120862024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120872024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120882024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
120892024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
120902024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
120912024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
120922024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
120932024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
120942024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
120952024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
120962024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
120972024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
120982024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120992024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121002024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121012024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121022024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
121032024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
121042024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
121052024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
121062024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121072024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121082024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
121092024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
121102024-11-14T04:57:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
121112024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
121122024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
121132024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
121142024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
121152024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
121162024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121172024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
121182024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
121192024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
121202024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121212024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
121222024-11-14T04:57:45.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
121232024-11-14T04:57:45.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121242024-11-14T04:57:45.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121252024-11-14T04:57:45.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121262024-11-14T04:57:45.319ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.387577ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
121272024-11-14T04:57:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
121282024-11-14T04:57:45.321ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
121292024-11-14T04:57:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121302024-11-14T04:57:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
121312024-11-14T04:57:45.321ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121322024-11-14T04:57:45.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121332024-11-14T04:57:45.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
121342024-11-14T04:57:45.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121352024-11-14T04:57:45.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121362024-11-14T04:57:45.326ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 103557597 } })
121372024-11-14T04:57:45.326ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
121382024-11-14T04:57:45.329ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 106351438 } })
121392024-11-14T04:57:45.329ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
121402024-11-14T04:57:45.334ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121412024-11-14T04:57:45.335ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
121422024-11-14T04:57:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121432024-11-14T04:57:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121442024-11-14T04:57:45.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121452024-11-14T04:57:45.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121462024-11-14T04:57:45.384ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121472024-11-14T04:57:45.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121482024-11-14T04:57:45.385ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121492024-11-14T04:57:45.386ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121502024-11-14T04:57:45.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121512024-11-14T04:57:45.387ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 71.430578ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
121522024-11-14T04:57:45.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
121532024-11-14T04:57:45.388ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
121542024-11-14T04:57:45.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
121552024-11-14T04:57:45.389ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121562024-11-14T04:57:45.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121572024-11-14T04:57:45.392ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121582024-11-14T04:57:45.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121592024-11-14T04:57:45.393ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 170432603 } })
121602024-11-14T04:57:45.393ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
121612024-11-14T04:57:45.394ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.227736ms iteration = 24 status = {"errors":[],"records_set_to_done":[]}
121622024-11-14T04:57:45.394ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121632024-11-14T04:57:45.395ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
121642024-11-14T04:57:45.395ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
121652024-11-14T04:57:45.395ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
121662024-11-14T04:57:45.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
121672024-11-14T04:57:45.397ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121682024-11-14T04:57:45.398ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
121692024-11-14T04:57:45.398ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121702024-11-14T04:57:45.398ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121712024-11-14T04:57:45.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121722024-11-14T04:57:45.400ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
121732024-11-14T04:57:45.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
121742024-11-14T04:57:45.400ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
121752024-11-14T04:57:45.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121762024-11-14T04:57:45.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
121772024-11-14T04:57:45.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
121782024-11-14T04:57:45.401ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
121792024-11-14T04:57:45.401ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121802024-11-14T04:57:45.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
121812024-11-14T04:57:45.403ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
121822024-11-14T04:57:45.403ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121832024-11-14T04:57:45.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121842024-11-14T04:57:45.407ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 92.232761ms iteration = 24 status = 0
121852024-11-14T04:57:45.408ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121862024-11-14T04:57:45.411ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
121872024-11-14T04:57:45.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121882024-11-14T04:57:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121892024-11-14T04:57:45.422ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
121902024-11-14T04:57:45.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
121912024-11-14T04:57:45.425ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
121922024-11-14T04:57:45.425ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
121932024-11-14T04:57:45.425ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b377aa67-9199-47be-b10d-8b2bb8b4a7be", "content-length": "26", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
121942024-11-14T04:57:45.425ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
121952024-11-14T04:57:45.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121962024-11-14T04:57:45.428ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
121972024-11-14T04:57:45.429ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
121982024-11-14T04:57:45.429ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
121992024-11-14T04:57:45.430ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 115.296339ms iteration = 27 status = {"physical_disks_added":0}
122002024-11-14T04:57:45.430ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "69ac9537-529b-472c-b696-50bf04beaa1c", "content-length": "26", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
122012024-11-14T04:57:45.430ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
122022024-11-14T04:57:45.432ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
122032024-11-14T04:57:45.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
122042024-11-14T04:57:45.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122052024-11-14T04:57:45.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122062024-11-14T04:57:45.439ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
122072024-11-14T04:57:45.439ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
122082024-11-14T04:57:45.439ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
122092024-11-14T04:57:45.439ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
122102024-11-14T04:57:45.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
122112024-11-14T04:57:45.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
122122024-11-14T04:57:45.442ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 126.173854ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
122132024-11-14T04:57:45.451ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 136.566554ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
122142024-11-14T04:57:45.453ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
122152024-11-14T04:57:45.453ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 138.328206ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
122162024-11-14T04:57:45.461ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 145.39102ms iteration = 24 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
122172024-11-14T04:57:45.472ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 157.585894ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
122182024-11-14T04:57:45.479ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
122192024-11-14T04:57:45.479ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 164.524774ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
122202024-11-14T04:57:45.493ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
122212024-11-14T04:57:45.493ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
122222024-11-14T04:57:45.493ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 178.738576ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
122232024-11-14T04:57:45.496ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122242024-11-14T04:57:45.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
122252024-11-14T04:57:45.499ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
122262024-11-14T04:57:45.499ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
122272024-11-14T04:57:45.499ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "af9317ba-997b-4ee9-8e05-3d8d7cf410f2", "content-length": "26", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
122282024-11-14T04:57:45.499ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
122292024-11-14T04:57:45.501ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
122302024-11-14T04:57:45.623ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
122312024-11-14T04:57:45.623ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "47b0217e-c43a-4a17-8e32-a435ba6d22e7", "content-length": "1", "date": "Thu, 14 Nov 2024 04:57:44 GMT"} })
122322024-11-14T04:57:45.625ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
122332024-11-14T04:57:45.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
122342024-11-14T04:57:45.628ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 313.577761ms iteration = 24 status = null
122352024-11-14T04:57:45.631ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 316.952609ms iteration = 24 status = {}
122362024-11-14T04:57:45.632ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122372024-11-14T04:57:45.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
122382024-11-14T04:57:45.634ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122392024-11-14T04:57:45.636ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
122402024-11-14T04:57:45.636ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122412024-11-14T04:57:45.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
122422024-11-14T04:57:45.638ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122432024-11-14T04:57:45.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
122442024-11-14T04:57:45.640ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122452024-11-14T04:57:45.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
122462024-11-14T04:57:45.640ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
122472024-11-14T04:57:45.640ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
122482024-11-14T04:57:45.640ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "dab49848-e54d-4221-8616-3e9df762826d", "content-length": "372", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
122492024-11-14T04:57:45.640ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122502024-11-14T04:57:45.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122512024-11-14T04:57:45.658ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
122522024-11-14T04:57:45.658ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
122532024-11-14T04:57:45.658ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "46eebde9-252f-4667-a72e-61f9314ca399", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
122542024-11-14T04:57:45.658ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122552024-11-14T04:57:45.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122562024-11-14T04:57:45.661ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122572024-11-14T04:57:45.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122582024-11-14T04:57:45.665ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122592024-11-14T04:57:45.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122602024-11-14T04:57:45.670ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
122612024-11-14T04:57:45.670ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
122622024-11-14T04:57:45.670ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5f283c20-4036-4299-8f11-8b287e8ce525", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
122632024-11-14T04:57:45.672ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 357.42281ms iteration = 25 status = {}
122642024-11-14T04:57:45.702ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122652024-11-14T04:57:45.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122662024-11-14T04:57:45.705ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
122672024-11-14T04:57:45.705ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122682024-11-14T04:57:45.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122692024-11-14T04:57:45.719ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
122702024-11-14T04:57:45.719ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fd02eef-5330-4274-b315-3f47dbbc24f7", "content-length": "134", "date": "Thu, 14 Nov 2024 04:57:44 GMT"} })
122712024-11-14T04:57:45.719ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
122722024-11-14T04:57:45.719ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
122732024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
122742024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
122752024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
122762024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
122772024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
122782024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
122792024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
122802024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
122812024-11-14T04:57:45.720ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
122822024-11-14T04:57:45.720ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122832024-11-14T04:57:45.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122842024-11-14T04:57:45.723ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
122852024-11-14T04:57:45.723ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "2237c7f7-6a31-4c67-8fa2-124a68b57a0f", "content-length": "2", "date": "Thu, 14 Nov 2024 04:57:44 GMT"} })
122862024-11-14T04:57:45.723ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
122872024-11-14T04:57:45.723ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1fb99ef-bef9-438c-ab44-f3ed12c2ea52", "content-length": "51", "date": "Thu, 14 Nov 2024 04:57:44 GMT"} })
122882024-11-14T04:57:45.723ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
122892024-11-14T04:57:45.723ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
122902024-11-14T04:57:45.763ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
122912024-11-14T04:57:45.851ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ee4ab46-6b0a-4978-bd3d-9d3f54560d88", "content-length": "691", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
122922024-11-14T04:57:45.851ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
122932024-11-14T04:57:45.851ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
122942024-11-14T04:57:45.851ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
122952024-11-14T04:57:45.852ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 537.948619ms iteration = 24 status = {}
122962024-11-14T04:57:46.059ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "634049ce-457f-4e1b-ae8c-7e0d62f1ca7d", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
122972024-11-14T04:57:46.059ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
122982024-11-14T04:57:46.060ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
122992024-11-14T04:57:46.060ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "be7525e6-d619-4398-90aa-69c949dfbf4d", "content-length": "682", "date": "Thu, 14 Nov 2024 04:57:45 GMT"} })
123002024-11-14T04:57:46.060ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
123012024-11-14T04:57:46.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
123022024-11-14T04:57:46.064ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
123032024-11-14T04:57:46.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
123042024-11-14T04:57:46.067ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
123052024-11-14T04:57:46.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
123062024-11-14T04:57:46.069ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
123072024-11-14T04:57:46.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
123082024-11-14T04:57:46.076ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 761.632246ms iteration = 24 status = {}
123092024-11-14T04:57:50.256ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1082 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = d40a1885-af6d-4581-a335-a69525adbe2f response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
123102024-11-14T04:57:55.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
123112024-11-14T04:57:55.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
123122024-11-14T04:57:55.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123132024-11-14T04:57:55.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
123142024-11-14T04:57:55.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
123152024-11-14T04:57:55.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123162024-11-14T04:57:55.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123172024-11-14T04:57:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
123182024-11-14T04:57:55.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123192024-11-14T04:57:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123202024-11-14T04:57:55.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123212024-11-14T04:57:55.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123222024-11-14T04:57:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123232024-11-14T04:57:55.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
123242024-11-14T04:57:55.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123252024-11-14T04:57:55.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123262024-11-14T04:57:55.338ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
123272024-11-14T04:57:55.338ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.031607ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
123282024-11-14T04:57:55.344ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
123292024-11-14T04:57:55.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
123302024-11-14T04:57:55.346ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.015698ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
123312024-11-14T04:57:57.063ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1:0]:49931
123322024-11-14T04:57:57.064ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:49931 req_id = 995eae27-22b1-4b57-82a9-e4bd4ed6e837 roles = RoleSet { roles: {} } uri = /metrics/producers
123332024-11-14T04:57:57.064ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:49931 req_id = 995eae27-22b1-4b57-82a9-e4bd4ed6e837 resource = Database result = Ok(()) uri = /metrics/producers
123342024-11-14T04:57:57.100ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = f872674e-3abd-4856-abe7-9936da747fcf
123352024-11-14T04:57:57.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35988 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:49931 req_id = 995eae27-22b1-4b57-82a9-e4bd4ed6e837 response_code = 201 uri = /metrics/producers
123362024-11-14T04:58:05.257ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1046 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 818bf33d-ed2e-4b24-a236-d6a5368fb0b8 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
123372024-11-14T04:58:05.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
123382024-11-14T04:58:05.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
123392024-11-14T04:58:05.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123402024-11-14T04:58:05.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
123412024-11-14T04:58:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123422024-11-14T04:58:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
123432024-11-14T04:58:05.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123442024-11-14T04:58:05.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
123452024-11-14T04:58:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
123462024-11-14T04:58:05.319ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.297185ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
123472024-11-14T04:58:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
123482024-11-14T04:58:05.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123492024-11-14T04:58:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123502024-11-14T04:58:05.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123512024-11-14T04:58:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123522024-11-14T04:58:05.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123532024-11-14T04:58:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
123542024-11-14T04:58:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123552024-11-14T04:58:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123562024-11-14T04:58:05.334ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
123572024-11-14T04:58:05.334ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.200736ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
123582024-11-14T04:58:10.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 1ffd3722-38fc-4d2e-9fc0-31f63890a0aa roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
123592024-11-14T04:58:10.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 1ffd3722-38fc-4d2e-9fc0-31f63890a0aa resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
123602024-11-14T04:58:10.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3210 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 1ffd3722-38fc-4d2e-9fc0-31f63890a0aa response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
123612024-11-14T04:58:10.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = f4127483-6ea3-4357-ae5f-3671a2813be9 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
123622024-11-14T04:58:10.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = f4127483-6ea3-4357-ae5f-3671a2813be9 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
123632024-11-14T04:58:10.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2425 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = f4127483-6ea3-4357-ae5f-3671a2813be9 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
123642024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
123652024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
123662024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123672024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
123682024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
123692024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
123702024-11-14T04:58:15.315ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
123712024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 20.191\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
123722024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
123732024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
123742024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
123752024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
123762024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
123772024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
123782024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
123792024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
123802024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
123812024-11-14T04:58:15.315ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
123822024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123832024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
123842024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
123852024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
123862024-11-14T04:58:15.315ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:48:15.315240806 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
123872024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
123882024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
123892024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
123902024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
123912024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
123922024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
123932024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
123942024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
123952024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
123962024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
123972024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
123982024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
123992024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
124002024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
124012024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
124022024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
124032024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
124042024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
124052024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
124062024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
124072024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
124082024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
124092024-11-14T04:58:15.315ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
124102024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
124112024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
124122024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
124132024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
124142024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
124152024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124162024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
124172024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
124182024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
124192024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
124202024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
124212024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
124222024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
124232024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
124242024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
124252024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
124262024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
124272024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
124282024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124292024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124302024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
124312024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
124322024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
124332024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124342024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
124352024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
124362024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
124372024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
124382024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
124392024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
124402024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
124412024-11-14T04:58:15.315ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 93425411 } })
124422024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
124432024-11-14T04:58:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
124442024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
124452024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
124462024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
124472024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
124482024-11-14T04:58:15.316ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 93495163 } })
124492024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
124502024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124512024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
124522024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
124532024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124542024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124552024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
124562024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
124572024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
124582024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124592024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124602024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124612024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124622024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
124632024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.269567ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
124642024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124652024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
124662024-11-14T04:58:15.316ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.301198ms iteration = 14 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
124672024-11-14T04:58:15.317ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124682024-11-14T04:58:15.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124692024-11-14T04:58:15.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124702024-11-14T04:58:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.003536ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
124712024-11-14T04:58:15.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
124722024-11-14T04:58:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
124732024-11-14T04:58:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
124742024-11-14T04:58:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124752024-11-14T04:58:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124762024-11-14T04:58:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
124772024-11-14T04:58:15.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124782024-11-14T04:58:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
124792024-11-14T04:58:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
124802024-11-14T04:58:15.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
124812024-11-14T04:58:15.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
124822024-11-14T04:58:15.322ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
124832024-11-14T04:58:15.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
124842024-11-14T04:58:15.323ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
124852024-11-14T04:58:15.323ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124862024-11-14T04:58:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
124872024-11-14T04:58:15.326ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:48:15.315240806 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
124882024-11-14T04:58:15.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
124892024-11-14T04:58:15.326ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 11.05076ms iteration = 13 status = {"expiration":"2024-11-14T04:48:15.315240806Z","pruned":[]}
124902024-11-14T04:58:15.326ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
124912024-11-14T04:58:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
124922024-11-14T04:58:15.328ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 106164200 } })
124932024-11-14T04:58:15.329ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
124942024-11-14T04:58:15.330ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124952024-11-14T04:58:15.330ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 108318212 } })
124962024-11-14T04:58:15.331ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
124972024-11-14T04:58:15.333ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124982024-11-14T04:58:15.333ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.98846ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
124992024-11-14T04:58:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
125002024-11-14T04:58:15.333ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
125012024-11-14T04:58:15.334ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 18.482645ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
125022024-11-14T04:58:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
125032024-11-14T04:58:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125042024-11-14T04:58:15.336ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125052024-11-14T04:58:15.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125062024-11-14T04:58:15.343ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125072024-11-14T04:58:15.346ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125082024-11-14T04:58:15.346ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125092024-11-14T04:58:15.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125102024-11-14T04:58:15.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125112024-11-14T04:58:15.355ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125122024-11-14T04:58:15.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125132024-11-14T04:58:15.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
125142024-11-14T04:58:15.375ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125152024-11-14T04:58:15.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
125162024-11-14T04:58:15.400ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
125172024-11-14T04:58:15.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
125182024-11-14T04:58:15.406ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
125192024-11-14T04:58:15.406ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
125202024-11-14T04:58:15.406ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
125212024-11-14T04:58:15.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
125222024-11-14T04:58:15.406ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
125232024-11-14T04:58:15.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
125242024-11-14T04:58:15.407ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125252024-11-14T04:58:15.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125262024-11-14T04:58:15.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
125272024-11-14T04:58:15.411ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
125282024-11-14T04:58:15.411ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
125292024-11-14T04:58:15.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
125302024-11-14T04:58:15.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
125312024-11-14T04:58:15.414ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
125322024-11-14T04:58:15.415ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125332024-11-14T04:58:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
125342024-11-14T04:58:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
125352024-11-14T04:58:15.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125362024-11-14T04:58:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125372024-11-14T04:58:15.417ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125382024-11-14T04:58:15.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
125392024-11-14T04:58:15.417ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
125402024-11-14T04:58:15.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
125412024-11-14T04:58:15.418ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
125422024-11-14T04:58:15.418ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
125432024-11-14T04:58:15.418ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 102.709294ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
125442024-11-14T04:58:15.420ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125452024-11-14T04:58:15.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125462024-11-14T04:58:15.420ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125472024-11-14T04:58:15.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125482024-11-14T04:58:15.421ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125492024-11-14T04:58:15.422ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
125502024-11-14T04:58:15.422ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
125512024-11-14T04:58:15.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
125522024-11-14T04:58:15.423ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 107.942375ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
125532024-11-14T04:58:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
125542024-11-14T04:58:15.424ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125552024-11-14T04:58:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125562024-11-14T04:58:15.425ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
125572024-11-14T04:58:15.425ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 110.253043ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
125582024-11-14T04:58:15.425ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
125592024-11-14T04:58:15.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
125602024-11-14T04:58:15.434ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 119.120529ms iteration = 25 status = {"errors":[],"records_set_to_done":[]}
125612024-11-14T04:58:15.434ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
125622024-11-14T04:58:15.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
125632024-11-14T04:58:15.436ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125642024-11-14T04:58:15.436ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 214448506 } })
125652024-11-14T04:58:15.437ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
125662024-11-14T04:58:15.437ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 122.192798ms iteration = 25 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
125672024-11-14T04:58:15.438ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
125682024-11-14T04:58:15.438ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
125692024-11-14T04:58:15.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
125702024-11-14T04:58:15.447ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
125712024-11-14T04:58:15.447ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 132.427755ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
125722024-11-14T04:58:15.450ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
125732024-11-14T04:58:15.450ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
125742024-11-14T04:58:15.450ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
125752024-11-14T04:58:15.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
125762024-11-14T04:58:15.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
125772024-11-14T04:58:15.458ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125782024-11-14T04:58:15.460ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125792024-11-14T04:58:15.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125802024-11-14T04:58:15.462ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125812024-11-14T04:58:15.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125822024-11-14T04:58:15.464ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125832024-11-14T04:58:15.464ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
125842024-11-14T04:58:15.464ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
125852024-11-14T04:58:15.464ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
125862024-11-14T04:58:15.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
125872024-11-14T04:58:15.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
125882024-11-14T04:58:15.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
125892024-11-14T04:58:15.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
125902024-11-14T04:58:15.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
125912024-11-14T04:58:15.466ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
125922024-11-14T04:58:15.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
125932024-11-14T04:58:15.468ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
125942024-11-14T04:58:15.468ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
125952024-11-14T04:58:15.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
125962024-11-14T04:58:15.471ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 156.129921ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
125972024-11-14T04:58:15.473ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
125982024-11-14T04:58:15.473ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
125992024-11-14T04:58:15.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126002024-11-14T04:58:15.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126012024-11-14T04:58:15.475ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 160.611871ms iteration = 6 status = {"nsuccess":5}
126022024-11-14T04:58:15.476ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
126032024-11-14T04:58:15.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
126042024-11-14T04:58:15.481ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126052024-11-14T04:58:15.482ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
126062024-11-14T04:58:15.482ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
126072024-11-14T04:58:15.482ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
126082024-11-14T04:58:15.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
126092024-11-14T04:58:15.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
126102024-11-14T04:58:15.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
126112024-11-14T04:58:15.484ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126122024-11-14T04:58:15.485ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
126132024-11-14T04:58:15.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
126142024-11-14T04:58:15.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
126152024-11-14T04:58:15.486ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
126162024-11-14T04:58:15.486ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
126172024-11-14T04:58:15.487ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2dee594-bc4d-42d0-b943-ca9dec75095a", "content-length": "26", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
126182024-11-14T04:58:15.487ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
126192024-11-14T04:58:15.487ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
126202024-11-14T04:58:15.487ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
126212024-11-14T04:58:15.487ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 172.592438ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
126222024-11-14T04:58:15.491ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
126232024-11-14T04:58:15.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
126242024-11-14T04:58:15.491ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126252024-11-14T04:58:15.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126262024-11-14T04:58:15.496ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126272024-11-14T04:58:15.499ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
126282024-11-14T04:58:15.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
126292024-11-14T04:58:15.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
126302024-11-14T04:58:15.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
126312024-11-14T04:58:15.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
126322024-11-14T04:58:15.506ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
126332024-11-14T04:58:15.506ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
126342024-11-14T04:58:15.506ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
126352024-11-14T04:58:15.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 190.869098ms iteration = 14 status = {"generation":1}
126362024-11-14T04:58:15.508ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
126372024-11-14T04:58:15.509ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 193.70302ms iteration = 25 status = 0
126382024-11-14T04:58:15.511ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
126392024-11-14T04:58:15.511ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
126402024-11-14T04:58:15.512ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a578909-021a-4459-aed1-03c358bf3e46", "content-length": "26", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
126412024-11-14T04:58:15.512ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
126422024-11-14T04:58:15.514ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
126432024-11-14T04:58:15.526ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126442024-11-14T04:58:15.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126452024-11-14T04:58:15.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126462024-11-14T04:58:15.527ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
126472024-11-14T04:58:15.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
126482024-11-14T04:58:15.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
126492024-11-14T04:58:15.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
126502024-11-14T04:58:15.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
126512024-11-14T04:58:15.534ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
126522024-11-14T04:58:15.553ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
126532024-11-14T04:58:15.553ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
126542024-11-14T04:58:15.553ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 238.183838ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
126552024-11-14T04:58:15.553ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
126562024-11-14T04:58:15.553ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 238.184557ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
126572024-11-14T04:58:15.558ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
126582024-11-14T04:58:15.558ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
126592024-11-14T04:58:15.559ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e515b3ed-f42a-43f2-8c04-b7cf79716987", "content-length": "26", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
126602024-11-14T04:58:15.559ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
126612024-11-14T04:58:15.561ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
126622024-11-14T04:58:15.565ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 249.794404ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
126632024-11-14T04:58:15.584ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
126642024-11-14T04:58:15.584ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
126652024-11-14T04:58:15.584ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
126662024-11-14T04:58:15.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
126672024-11-14T04:58:15.585ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126682024-11-14T04:58:15.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
126692024-11-14T04:58:15.589ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
126702024-11-14T04:58:15.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
126712024-11-14T04:58:15.592ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126722024-11-14T04:58:15.594ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
126732024-11-14T04:58:15.594ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
126742024-11-14T04:58:15.594ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
126752024-11-14T04:58:15.594ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 279.126433ms iteration = 14 status = {"generation":2}
126762024-11-14T04:58:15.594ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
126772024-11-14T04:58:15.594ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126782024-11-14T04:58:15.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126792024-11-14T04:58:15.606ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 291.289415ms iteration = 28 status = {"physical_disks_added":0}
126802024-11-14T04:58:15.637ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
126812024-11-14T04:58:15.637ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "49993866-90d7-4bc4-aec0-f5af6d0ace94", "content-length": "2", "date": "Thu, 14 Nov 2024 04:58:14 GMT"} })
126822024-11-14T04:58:15.639ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
126832024-11-14T04:58:15.639ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 324.285691ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
126842024-11-14T04:58:15.641ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
126852024-11-14T04:58:15.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
126862024-11-14T04:58:15.647ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
126872024-11-14T04:58:15.663ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 348.247055ms iteration = 25 status = {}
126882024-11-14T04:58:15.677ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
126892024-11-14T04:58:15.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
126902024-11-14T04:58:15.699ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
126912024-11-14T04:58:15.700ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "2d448b03-c0f1-40ea-9263-c64945302641", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
126922024-11-14T04:58:15.700ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
126932024-11-14T04:58:15.702ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "d4166fd6-2494-44f4-9d17-2f3c5405ee89", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
126942024-11-14T04:58:15.702ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
126952024-11-14T04:58:15.704ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
126962024-11-14T04:58:15.704ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 389.404387ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
126972024-11-14T04:58:15.718ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
126982024-11-14T04:58:15.718ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
126992024-11-14T04:58:15.718ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc10c70b-b99e-4a79-b0e0-36582f00bc74", "content-length": "372", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
127002024-11-14T04:58:15.718ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127012024-11-14T04:58:15.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127022024-11-14T04:58:15.720ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 405.367749ms iteration = 25 status = null
127032024-11-14T04:58:15.733ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
127042024-11-14T04:58:15.734ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "c9a3d8a1-d0cb-468f-ba66-36d4c062eeba", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
127052024-11-14T04:58:15.734ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
127062024-11-14T04:58:15.736ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1399131b-8bcc-4d8f-8062-e0d4f2c8ce43", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
127072024-11-14T04:58:15.736ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
127082024-11-14T04:58:15.738ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a593530a-1e35-4c23-9fe5-0cea8754fd1a", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
127092024-11-14T04:58:15.738ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
127102024-11-14T04:58:15.740ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
127112024-11-14T04:58:15.740ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 425.106591ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
127122024-11-14T04:58:15.748ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127132024-11-14T04:58:15.750ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
127142024-11-14T04:58:15.750ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
127152024-11-14T04:58:15.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
127162024-11-14T04:58:15.752ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127172024-11-14T04:58:15.753ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
127182024-11-14T04:58:15.753ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
127192024-11-14T04:58:15.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
127202024-11-14T04:58:15.765ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
127212024-11-14T04:58:15.765ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
127222024-11-14T04:58:15.765ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a5687675-c71e-4b05-852c-d4c056810890", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
127232024-11-14T04:58:15.765ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127242024-11-14T04:58:15.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127252024-11-14T04:58:15.768ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127262024-11-14T04:58:15.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127272024-11-14T04:58:15.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127282024-11-14T04:58:15.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127292024-11-14T04:58:15.785ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
127302024-11-14T04:58:15.785ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
127312024-11-14T04:58:15.785ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "64be32cf-67bb-498e-8fc1-675f3b9ccbb5", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
127322024-11-14T04:58:15.787ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 472.586555ms iteration = 26 status = {}
127332024-11-14T04:58:15.788ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127342024-11-14T04:58:15.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127352024-11-14T04:58:15.791ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
127362024-11-14T04:58:15.791ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127372024-11-14T04:58:15.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127382024-11-14T04:58:15.801ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
127392024-11-14T04:58:15.802ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b01269c-006e-43bc-a3ce-0c842cc389fd", "content-length": "134", "date": "Thu, 14 Nov 2024 04:58:14 GMT"} })
127402024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
127412024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
127422024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
127432024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
127442024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
127452024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
127462024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
127472024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
127482024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
127492024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
127502024-11-14T04:58:15.802ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
127512024-11-14T04:58:15.803ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127522024-11-14T04:58:15.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127532024-11-14T04:58:15.805ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
127542024-11-14T04:58:15.805ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6d14dc3-0a54-40bf-a589-a663bc6858a0", "content-length": "2", "date": "Thu, 14 Nov 2024 04:58:14 GMT"} })
127552024-11-14T04:58:15.805ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
127562024-11-14T04:58:15.805ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b917a80-b512-406e-b7ef-993e2bc76217", "content-length": "51", "date": "Thu, 14 Nov 2024 04:58:14 GMT"} })
127572024-11-14T04:58:15.805ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
127582024-11-14T04:58:15.805ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
127592024-11-14T04:58:15.851ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
127602024-11-14T04:58:15.915ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e7720ef1-1b49-4a34-82d3-7f474544a731", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
127612024-11-14T04:58:15.915ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
127622024-11-14T04:58:15.915ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
127632024-11-14T04:58:15.916ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3f4722f-9192-485f-9b3d-ddab68216741", "content-length": "682", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
127642024-11-14T04:58:15.916ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127652024-11-14T04:58:15.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127662024-11-14T04:58:15.916ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3743af40-bde9-4f0d-aa4a-31228a4436e5", "content-length": "691", "date": "Thu, 14 Nov 2024 04:58:15 GMT"} })
127672024-11-14T04:58:15.916ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
127682024-11-14T04:58:15.916ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
127692024-11-14T04:58:15.916ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
127702024-11-14T04:58:15.919ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 604.028711ms iteration = 25 status = {}
127712024-11-14T04:58:15.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127722024-11-14T04:58:15.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127732024-11-14T04:58:15.924ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127742024-11-14T04:58:15.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127752024-11-14T04:58:15.926ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127762024-11-14T04:58:15.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127772024-11-14T04:58:15.932ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 617.362038ms iteration = 25 status = {}
127782024-11-14T04:58:20.257ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1283 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = a86dc347-9085-4217-a50b-5a60f6867e17 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
127792024-11-14T04:58:25.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 1c88753a-52c3-468d-baf6-291d1b584396 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
127802024-11-14T04:58:25.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 1c88753a-52c3-468d-baf6-291d1b584396 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
127812024-11-14T04:58:25.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3177 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 1c88753a-52c3-468d-baf6-291d1b584396 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
127822024-11-14T04:58:25.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 0e866796-0c16-4869-9037-9ba5f7e7d6fd roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
127832024-11-14T04:58:25.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 0e866796-0c16-4869-9037-9ba5f7e7d6fd resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
127842024-11-14T04:58:25.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3004 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 0e866796-0c16-4869-9037-9ba5f7e7d6fd response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
127852024-11-14T04:58:25.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
127862024-11-14T04:58:25.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
127872024-11-14T04:58:25.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
127882024-11-14T04:58:25.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
127892024-11-14T04:58:25.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
127902024-11-14T04:58:25.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
127912024-11-14T04:58:25.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127922024-11-14T04:58:25.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
127932024-11-14T04:58:25.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
127942024-11-14T04:58:25.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
127952024-11-14T04:58:25.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
127962024-11-14T04:58:25.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
127972024-11-14T04:58:25.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127982024-11-14T04:58:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
127992024-11-14T04:58:25.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128002024-11-14T04:58:25.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128012024-11-14T04:58:25.335ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
128022024-11-14T04:58:25.335ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.601736ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
128032024-11-14T04:58:25.343ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128042024-11-14T04:58:25.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128052024-11-14T04:58:25.384ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.956336ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
128062024-11-14T04:58:35.256ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1147 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = da963574-9a64-4abb-ae99-e0e171f56e9a response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
128072024-11-14T04:58:35.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
128082024-11-14T04:58:35.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128092024-11-14T04:58:35.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
128102024-11-14T04:58:35.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128112024-11-14T04:58:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128122024-11-14T04:58:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128132024-11-14T04:58:35.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128142024-11-14T04:58:35.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128152024-11-14T04:58:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128162024-11-14T04:58:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
128172024-11-14T04:58:35.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128182024-11-14T04:58:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128192024-11-14T04:58:35.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128202024-11-14T04:58:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128212024-11-14T04:58:35.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128222024-11-14T04:58:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
128232024-11-14T04:58:35.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128242024-11-14T04:58:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128252024-11-14T04:58:35.336ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
128262024-11-14T04:58:35.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.41675ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
128272024-11-14T04:58:35.345ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.310057ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
128282024-11-14T04:58:40.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = a377c544-393b-47e7-91a8-c485977c475f roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
128292024-11-14T04:58:40.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = a377c544-393b-47e7-91a8-c485977c475f resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
128302024-11-14T04:58:40.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3591 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = a377c544-393b-47e7-91a8-c485977c475f response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
128312024-11-14T04:58:40.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = c100d2ed-aec1-49b9-ae3f-6adb35ed29bc roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
128322024-11-14T04:58:40.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = c100d2ed-aec1-49b9-ae3f-6adb35ed29bc resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
128332024-11-14T04:58:40.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2581 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = c100d2ed-aec1-49b9-ae3f-6adb35ed29bc response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
128342024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
128352024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
128362024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128372024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
128382024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
128392024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
128402024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
128412024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
128422024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
128432024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
128442024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
128452024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
128462024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
128472024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
128482024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
128492024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
128502024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
128512024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
128522024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128532024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
128542024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
128552024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
128562024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
128572024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
128582024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
128592024-11-14T04:58:45.314ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
128602024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128612024-11-14T04:58:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
128622024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128632024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
128642024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
128652024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
128662024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
128672024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
128682024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
128692024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
128702024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
128712024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
128722024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
128732024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
128742024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
128752024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
128762024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128772024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
128782024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
128792024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
128802024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128812024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
128822024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
128832024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
128842024-11-14T04:58:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
128852024-11-14T04:58:45.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128862024-11-14T04:58:45.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128872024-11-14T04:58:45.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128882024-11-14T04:58:45.318ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 96220719 } })
128892024-11-14T04:58:45.318ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.1651ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
128902024-11-14T04:58:45.318ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
128912024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
128922024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128932024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
128942024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
128952024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
128962024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
128972024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128982024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
128992024-11-14T04:58:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
129002024-11-14T04:58:45.323ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
129012024-11-14T04:58:45.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
129022024-11-14T04:58:45.330ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 108397792 } })
129032024-11-14T04:58:45.331ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
129042024-11-14T04:58:45.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
129052024-11-14T04:58:45.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
129062024-11-14T04:58:45.336ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
129072024-11-14T04:58:45.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
129082024-11-14T04:58:45.381ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
129092024-11-14T04:58:45.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
129102024-11-14T04:58:45.386ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 71.294874ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
129112024-11-14T04:58:45.386ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129122024-11-14T04:58:45.387ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129132024-11-14T04:58:45.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
129142024-11-14T04:58:45.389ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129152024-11-14T04:58:45.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
129162024-11-14T04:58:45.390ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129172024-11-14T04:58:45.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
129182024-11-14T04:58:45.390ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
129192024-11-14T04:58:45.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
129202024-11-14T04:58:45.391ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
129212024-11-14T04:58:45.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
129222024-11-14T04:58:45.392ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
129232024-11-14T04:58:45.392ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
129242024-11-14T04:58:45.392ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
129252024-11-14T04:58:45.392ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
129262024-11-14T04:58:45.393ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb55a417-2e8f-4235-a55c-9f4bf5e35212", "content-length": "26", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
129272024-11-14T04:58:45.393ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
129282024-11-14T04:58:45.393ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f0d3b01-f8b7-45dd-b398-3010b3ba0e46", "content-length": "26", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
129292024-11-14T04:58:45.393ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
129302024-11-14T04:58:45.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
129312024-11-14T04:58:45.393ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
129322024-11-14T04:58:45.393ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129332024-11-14T04:58:45.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
129342024-11-14T04:58:45.394ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129352024-11-14T04:58:45.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
129362024-11-14T04:58:45.396ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
129372024-11-14T04:58:45.396ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
129382024-11-14T04:58:45.396ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
129392024-11-14T04:58:45.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
129402024-11-14T04:58:45.396ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
129412024-11-14T04:58:45.396ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 82.295723ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
129422024-11-14T04:58:45.396ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 174697352 } })
129432024-11-14T04:58:45.397ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
129442024-11-14T04:58:45.397ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
129452024-11-14T04:58:45.398ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
129462024-11-14T04:58:45.398ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
129472024-11-14T04:58:45.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
129482024-11-14T04:58:45.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
129492024-11-14T04:58:45.401ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129502024-11-14T04:58:45.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
129512024-11-14T04:58:45.405ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
129522024-11-14T04:58:45.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
129532024-11-14T04:58:45.406ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 91.84485ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
129542024-11-14T04:58:45.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129552024-11-14T04:58:45.413ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
129562024-11-14T04:58:45.413ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
129572024-11-14T04:58:45.413ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
129582024-11-14T04:58:45.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
129592024-11-14T04:58:45.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
129602024-11-14T04:58:45.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
129612024-11-14T04:58:45.416ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 102.450687ms iteration = 26 status = 0
129622024-11-14T04:58:45.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
129632024-11-14T04:58:45.418ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
129642024-11-14T04:58:45.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
129652024-11-14T04:58:45.419ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 104.64545ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
129662024-11-14T04:58:45.439ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
129672024-11-14T04:58:45.439ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 124.829345ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
129682024-11-14T04:58:45.446ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 131.594351ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
129692024-11-14T04:58:45.449ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129702024-11-14T04:58:45.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
129712024-11-14T04:58:45.452ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129722024-11-14T04:58:45.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
129732024-11-14T04:58:45.460ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
129742024-11-14T04:58:45.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
129752024-11-14T04:58:45.475ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 161.042673ms iteration = 26 status = {}
129762024-11-14T04:58:45.476ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 161.271441ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
129772024-11-14T04:58:45.476ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
129782024-11-14T04:58:45.476ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
129792024-11-14T04:58:45.477ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7325eb3-a325-4723-8757-63a5a701cb8e", "content-length": "26", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
129802024-11-14T04:58:45.477ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
129812024-11-14T04:58:45.477ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
129822024-11-14T04:58:45.477ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
129832024-11-14T04:58:45.477ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
129842024-11-14T04:58:45.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
129852024-11-14T04:58:45.479ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
129862024-11-14T04:58:45.496ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
129872024-11-14T04:58:45.497ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9c6bc0b-2a6a-4939-9ce8-075264528e19", "content-length": "2", "date": "Thu, 14 Nov 2024 04:58:44 GMT"} })
129882024-11-14T04:58:45.497ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 182.33971ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
129892024-11-14T04:58:45.498ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
129902024-11-14T04:58:45.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
129912024-11-14T04:58:45.500ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 186.260944ms iteration = 26 status = null
129922024-11-14T04:58:45.501ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
129932024-11-14T04:58:45.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
129942024-11-14T04:58:45.525ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
129952024-11-14T04:58:45.525ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
129962024-11-14T04:58:45.525ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 210.852226ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
129972024-11-14T04:58:45.555ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
129982024-11-14T04:58:45.555ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
129992024-11-14T04:58:45.556ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "79993f9d-ee25-467c-a983-e5f2008b3b59", "content-length": "372", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
130002024-11-14T04:58:45.556ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
130012024-11-14T04:58:45.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
130022024-11-14T04:58:45.571ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 256.675273ms iteration = 29 status = {"physical_disks_added":0}
130032024-11-14T04:58:45.604ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
130042024-11-14T04:58:45.604ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
130052024-11-14T04:58:45.604ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "847dd8c3-b888-462a-af8a-f152acaaff46", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
130062024-11-14T04:58:45.604ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
130072024-11-14T04:58:45.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
130082024-11-14T04:58:45.607ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
130092024-11-14T04:58:45.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
130102024-11-14T04:58:45.610ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
130112024-11-14T04:58:45.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
130122024-11-14T04:58:45.614ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
130132024-11-14T04:58:45.614ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
130142024-11-14T04:58:45.615ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e610a9eb-b498-41e0-a76a-c1f2aa304a2e", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
130152024-11-14T04:58:45.617ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 302.633114ms iteration = 27 status = {}
130162024-11-14T04:58:45.633ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130172024-11-14T04:58:45.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130182024-11-14T04:58:45.636ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
130192024-11-14T04:58:45.636ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130202024-11-14T04:58:45.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130212024-11-14T04:58:45.646ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
130222024-11-14T04:58:45.646ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b8fe7ae-212a-4ebc-b18c-685449b82669", "content-length": "134", "date": "Thu, 14 Nov 2024 04:58:44 GMT"} })
130232024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
130242024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
130252024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
130262024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
130272024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
130282024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
130292024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
130302024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
130312024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
130322024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
130332024-11-14T04:58:45.647ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
130342024-11-14T04:58:45.647ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130352024-11-14T04:58:45.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130362024-11-14T04:58:45.649ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
130372024-11-14T04:58:45.649ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b6b493c-47f4-4601-ae1c-466bacd1063d", "content-length": "2", "date": "Thu, 14 Nov 2024 04:58:44 GMT"} })
130382024-11-14T04:58:45.649ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
130392024-11-14T04:58:45.649ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "d87580d2-8047-4432-b461-76bfbf9f8b20", "content-length": "51", "date": "Thu, 14 Nov 2024 04:58:44 GMT"} })
130402024-11-14T04:58:45.649ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
130412024-11-14T04:58:45.649ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
130422024-11-14T04:58:45.657ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
130432024-11-14T04:58:45.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
130442024-11-14T04:58:45.687ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130452024-11-14T04:58:45.689ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
130462024-11-14T04:58:45.689ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
130472024-11-14T04:58:45.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
130482024-11-14T04:58:45.691ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130492024-11-14T04:58:45.692ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
130502024-11-14T04:58:45.692ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
130512024-11-14T04:58:45.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
130522024-11-14T04:58:45.787ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
130532024-11-14T04:58:45.787ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f78ea1d2-4bb4-457a-b1c3-4a93e25bee40", "content-length": "691", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
130542024-11-14T04:58:45.787ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
130552024-11-14T04:58:45.787ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
130562024-11-14T04:58:45.787ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
130572024-11-14T04:58:45.789ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 474.943084ms iteration = 26 status = {}
130582024-11-14T04:58:45.864ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6cd7778e-a454-4f0d-813c-d928ea8ca51b", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
130592024-11-14T04:58:45.865ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
130602024-11-14T04:58:45.865ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
130612024-11-14T04:58:45.869ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2f18bce-f493-4d99-aed9-e98b08edaad2", "content-length": "682", "date": "Thu, 14 Nov 2024 04:58:45 GMT"} })
130622024-11-14T04:58:45.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130632024-11-14T04:58:45.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130642024-11-14T04:58:45.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130652024-11-14T04:58:45.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130662024-11-14T04:58:45.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130672024-11-14T04:58:45.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130682024-11-14T04:58:45.879ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130692024-11-14T04:58:45.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130702024-11-14T04:58:45.886ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 571.692585ms iteration = 26 status = {}
130712024-11-14T04:58:50.257ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1083 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 014d7efc-5669-4e7e-a061-b236c4353ade response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
130722024-11-14T04:58:50.314ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
130732024-11-14T04:58:50.314ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
130742024-11-14T04:58:50.345ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
130752024-11-14T04:58:50.345ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:47439
130762024-11-14T04:58:50.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:47439 req_id = 6268e86d-00f3-40d6-bcbf-7c74c64e8b30 roles = RoleSet { roles: {} } uri = /metrics/producers
130772024-11-14T04:58:50.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:47439 req_id = 6268e86d-00f3-40d6-bcbf-7c74c64e8b30 resource = Database result = Ok(()) uri = /metrics/producers
130782024-11-14T04:58:50.383ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 13db289d-f9b6-41e9-b3bd-5ab4f3105694
130792024-11-14T04:58:50.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38309 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:47439 req_id = 6268e86d-00f3-40d6-bcbf-7c74c64e8b30 response_code = 201 uri = /metrics/producers
130802024-11-14T04:58:50.384ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "6268e86d-00f3-40d6-bcbf-7c74c64e8b30", "content-length": "41", "date": "Thu, 14 Nov 2024 04:58:50 GMT"} })
130812024-11-14T04:58:50.386ZDEBGnexus (producer-registration-task): registered with nexus successfully
130822024-11-14T04:58:50.386ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
130832024-11-14T04:58:55.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 63fa1aaf-bddc-4567-be77-419f32310780 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
130842024-11-14T04:58:55.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 63fa1aaf-bddc-4567-be77-419f32310780 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
130852024-11-14T04:58:55.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3449 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 63fa1aaf-bddc-4567-be77-419f32310780 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
130862024-11-14T04:58:55.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 571c63e4-e4b9-4de2-abc8-e4b23777e1b5 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
130872024-11-14T04:58:55.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 571c63e4-e4b9-4de2-abc8-e4b23777e1b5 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
130882024-11-14T04:58:55.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2476 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 571c63e4-e4b9-4de2-abc8-e4b23777e1b5 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
130892024-11-14T04:58:55.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
130902024-11-14T04:58:55.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
130912024-11-14T04:58:55.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
130922024-11-14T04:58:55.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130932024-11-14T04:58:55.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
130942024-11-14T04:58:55.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130952024-11-14T04:58:55.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130962024-11-14T04:58:55.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
130972024-11-14T04:58:55.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130982024-11-14T04:58:55.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130992024-11-14T04:58:55.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131002024-11-14T04:58:55.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
131012024-11-14T04:58:55.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131022024-11-14T04:58:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
131032024-11-14T04:58:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131042024-11-14T04:58:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
131052024-11-14T04:58:55.336ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
131062024-11-14T04:58:55.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.048688ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
131072024-11-14T04:58:55.342ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131082024-11-14T04:58:55.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131092024-11-14T04:58:55.370ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.508367ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
131102024-11-14T04:59:05.257ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1086 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 5681197e-ef21-4be8-aca8-8011f2a7ac6e response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
131112024-11-14T04:59:05.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
131122024-11-14T04:59:05.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131132024-11-14T04:59:05.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
131142024-11-14T04:59:05.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131152024-11-14T04:59:05.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
131162024-11-14T04:59:05.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131172024-11-14T04:59:05.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131182024-11-14T04:59:05.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131192024-11-14T04:59:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131202024-11-14T04:59:05.317ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.625045ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
131212024-11-14T04:59:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
131222024-11-14T04:59:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131232024-11-14T04:59:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
131242024-11-14T04:59:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131252024-11-14T04:59:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
131262024-11-14T04:59:05.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131272024-11-14T04:59:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
131282024-11-14T04:59:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131292024-11-14T04:59:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
131302024-11-14T04:59:05.332ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
131312024-11-14T04:59:05.332ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.280709ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
131322024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
131332024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131342024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
131352024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
131362024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
131372024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
131382024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
131392024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
131402024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
131412024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
131422024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
131432024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
131442024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
131452024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
131462024-11-14T04:59:15.314ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
131472024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
131482024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131492024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
131502024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
131512024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
131522024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131532024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
131542024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
131552024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
131562024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
131572024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
131582024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
131592024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
131602024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
131612024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131622024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
131632024-11-14T04:59:15.314ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 04:49:15.314772283 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
131642024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
131652024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
131662024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131672024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
131682024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
131692024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
131702024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
131712024-11-14T04:59:15.314ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
131722024-11-14T04:59:15.315ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
131732024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
131742024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
131752024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
131762024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
131772024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
131782024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
131792024-11-14T04:59:15.315ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7
131802024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
131812024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.601\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7"}
131822024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
131832024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
131842024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
131852024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
131862024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
131872024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
131882024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
131892024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
131902024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
131912024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
131922024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
131932024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
131942024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
131952024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
131962024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131972024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
131982024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
131992024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132002024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
132012024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
132022024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
132032024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132042024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
132052024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
132062024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132072024-11-14T04:59:15.315ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("48221b8a-d832-4d05-ae47-e92f81388c0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8c0d54f6-76b3-47cc-9828-b8e677cc2961.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 93406876 } })
132082024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
132092024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
132102024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
132112024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
132122024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
132132024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
132142024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
132152024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
132162024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
132172024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
132182024-11-14T04:59:15.315ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2327cea1-dd20-406b-af0b-11f432f90f24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("24da292b-72a6-4c9f-b124-daf5b3ed05d9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0f17ac9-6497-4365-ae94-118450acb817.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 93552370 } })
132192024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
132202024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
132212024-11-14T04:59:15.315ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.03878ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
132222024-11-14T04:59:15.316ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
132232024-11-14T04:59:15.316ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.134233ms iteration = 15 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
132242024-11-14T04:59:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
132252024-11-14T04:59:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132262024-11-14T04:59:15.318ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.45875ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
132272024-11-14T04:59:15.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132282024-11-14T04:59:15.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
132292024-11-14T04:59:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
132302024-11-14T04:59:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
132312024-11-14T04:59:15.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
132322024-11-14T04:59:15.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
132332024-11-14T04:59:15.322ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132342024-11-14T04:59:15.324ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 102085857 } })
132352024-11-14T04:59:15.324ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 102224521 } })
132362024-11-14T04:59:15.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132372024-11-14T04:59:15.324ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
132382024-11-14T04:59:15.324ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
132392024-11-14T04:59:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
132402024-11-14T04:59:15.327ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132412024-11-14T04:59:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132422024-11-14T04:59:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
132432024-11-14T04:59:15.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132442024-11-14T04:59:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132452024-11-14T04:59:15.329ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 04:49:15.314772283 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
132462024-11-14T04:59:15.329ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.954044ms iteration = 14 status = {"expiration":"2024-11-14T04:49:15.314772283Z","pruned":[]}
132472024-11-14T04:59:15.329ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
132482024-11-14T04:59:15.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
132492024-11-14T04:59:15.331ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132502024-11-14T04:59:15.333ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.248188ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
132512024-11-14T04:59:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
132522024-11-14T04:59:15.334ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
132532024-11-14T04:59:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
132542024-11-14T04:59:15.339ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132552024-11-14T04:59:15.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132562024-11-14T04:59:15.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132572024-11-14T04:59:15.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132582024-11-14T04:59:15.345ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
132592024-11-14T04:59:15.345ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 31.295556ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
132602024-11-14T04:59:15.386ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132612024-11-14T04:59:15.388ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
132622024-11-14T04:59:15.389ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
132632024-11-14T04:59:15.389ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
132642024-11-14T04:59:15.389ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132652024-11-14T04:59:15.389ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
132662024-11-14T04:59:15.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132672024-11-14T04:59:15.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
132682024-11-14T04:59:15.393ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
132692024-11-14T04:59:15.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
132702024-11-14T04:59:15.409ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 94.060114ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
132712024-11-14T04:59:15.410ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
132722024-11-14T04:59:15.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132732024-11-14T04:59:15.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
132742024-11-14T04:59:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
132752024-11-14T04:59:15.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132762024-11-14T04:59:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132772024-11-14T04:59:15.416ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 101.595972ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
132782024-11-14T04:59:15.418ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 103.816796ms iteration = 27 status = {"errors":[],"records_set_to_done":[]}
132792024-11-14T04:59:15.419ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
132802024-11-14T04:59:15.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
132812024-11-14T04:59:15.423ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132822024-11-14T04:59:15.423ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
132832024-11-14T04:59:15.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
132842024-11-14T04:59:15.425ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132852024-11-14T04:59:15.427ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
132862024-11-14T04:59:15.427ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132872024-11-14T04:59:15.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132882024-11-14T04:59:15.428ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
132892024-11-14T04:59:15.428ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
132902024-11-14T04:59:15.428ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 113.331502ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
132912024-11-14T04:59:15.428ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
132922024-11-14T04:59:15.428ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132932024-11-14T04:59:15.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132942024-11-14T04:59:15.431ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132952024-11-14T04:59:15.432ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132962024-11-14T04:59:15.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132972024-11-14T04:59:15.433ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
132982024-11-14T04:59:15.433ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
132992024-11-14T04:59:15.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
133002024-11-14T04:59:15.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133012024-11-14T04:59:15.437ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133022024-11-14T04:59:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
133032024-11-14T04:59:15.440ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133042024-11-14T04:59:15.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133052024-11-14T04:59:15.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
133062024-11-14T04:59:15.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133072024-11-14T04:59:15.442ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
133082024-11-14T04:59:15.442ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 128.12554ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
133092024-11-14T04:59:15.445ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
133102024-11-14T04:59:15.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
133112024-11-14T04:59:15.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
133122024-11-14T04:59:15.449ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
133132024-11-14T04:59:15.450ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
133142024-11-14T04:59:15.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
133152024-11-14T04:59:15.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
133162024-11-14T04:59:15.481ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
133172024-11-14T04:59:15.481ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
133182024-11-14T04:59:15.482ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "909f762b-c00e-4aff-b432-927e225d71ef", "content-length": "26", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
133192024-11-14T04:59:15.482ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
133202024-11-14T04:59:15.484ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
133212024-11-14T04:59:15.485ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
133222024-11-14T04:59:15.485ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
133232024-11-14T04:59:15.486ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 171.121265ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
133242024-11-14T04:59:15.490ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 175.932574ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
133252024-11-14T04:59:15.491ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
133262024-11-14T04:59:15.491ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
133272024-11-14T04:59:15.491ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
133282024-11-14T04:59:15.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
133292024-11-14T04:59:15.493ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 178.963643ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
133302024-11-14T04:59:15.501ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
133312024-11-14T04:59:15.501ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
133322024-11-14T04:59:15.501ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
133332024-11-14T04:59:15.501ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
133342024-11-14T04:59:15.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
133352024-11-14T04:59:15.506ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133362024-11-14T04:59:15.508ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
133372024-11-14T04:59:15.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
133382024-11-14T04:59:15.509ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
133392024-11-14T04:59:15.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
133402024-11-14T04:59:15.510ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
133412024-11-14T04:59:15.510ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
133422024-11-14T04:59:15.511ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
133432024-11-14T04:59:15.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
133442024-11-14T04:59:15.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
133452024-11-14T04:59:15.552ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
133462024-11-14T04:59:15.563ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
133472024-11-14T04:59:15.563ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
133482024-11-14T04:59:15.563ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "20e0bfd9-1e9b-427b-9c36-8725ef387c0d", "content-length": "372", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
133492024-11-14T04:59:15.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133502024-11-14T04:59:15.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133512024-11-14T04:59:15.566ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
133522024-11-14T04:59:15.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
133532024-11-14T04:59:15.567ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
133542024-11-14T04:59:15.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
133552024-11-14T04:59:15.567ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 345525307 } })
133562024-11-14T04:59:15.567ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
133572024-11-14T04:59:15.590ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
133582024-11-14T04:59:15.590ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
133592024-11-14T04:59:15.590ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "683302e7-ad83-4e02-bac3-bc4c963a68dc", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
133602024-11-14T04:59:15.590ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133612024-11-14T04:59:15.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133622024-11-14T04:59:15.593ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133632024-11-14T04:59:15.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133642024-11-14T04:59:15.594ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133652024-11-14T04:59:15.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
133662024-11-14T04:59:15.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133672024-11-14T04:59:15.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133682024-11-14T04:59:15.603ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
133692024-11-14T04:59:15.603ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
133702024-11-14T04:59:15.603ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5e6d8334-8034-4409-b679-34b8510280b2", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
133712024-11-14T04:59:15.605ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 290.850292ms iteration = 28 status = {}
133722024-11-14T04:59:15.608ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133732024-11-14T04:59:15.609ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 294.51782ms iteration = 27 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
133742024-11-14T04:59:15.611ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
133752024-11-14T04:59:15.612ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
133762024-11-14T04:59:15.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
133772024-11-14T04:59:15.613ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133782024-11-14T04:59:15.614ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 299.341519ms iteration = 27 status = 0
133792024-11-14T04:59:15.615ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133802024-11-14T04:59:15.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
133812024-11-14T04:59:15.617ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
133822024-11-14T04:59:15.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
133832024-11-14T04:59:15.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
133842024-11-14T04:59:15.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
133852024-11-14T04:59:15.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
133862024-11-14T04:59:15.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133872024-11-14T04:59:15.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
133882024-11-14T04:59:15.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
133892024-11-14T04:59:15.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
133902024-11-14T04:59:15.628ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
133912024-11-14T04:59:15.628ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
133922024-11-14T04:59:15.628ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 313.512309ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
133932024-11-14T04:59:15.644ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133942024-11-14T04:59:15.646ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
133952024-11-14T04:59:15.646ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133962024-11-14T04:59:15.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
133972024-11-14T04:59:15.648ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133982024-11-14T04:59:15.648ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 334.23658ms iteration = 27 status = {}
133992024-11-14T04:59:15.650ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
134002024-11-14T04:59:15.650ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
134012024-11-14T04:59:15.650ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "206a06ed-81f8-4e65-8bbe-81136a15ed78", "content-length": "26", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
134022024-11-14T04:59:15.650ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
134032024-11-14T04:59:15.651ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
134042024-11-14T04:59:15.651ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
134052024-11-14T04:59:15.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
134062024-11-14T04:59:15.653ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
134072024-11-14T04:59:15.658ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
134082024-11-14T04:59:15.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
134092024-11-14T04:59:15.660ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134102024-11-14T04:59:15.662ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
134112024-11-14T04:59:15.662ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
134122024-11-14T04:59:15.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
134132024-11-14T04:59:15.663ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
134142024-11-14T04:59:15.663ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 0 version = 2
134152024-11-14T04:59:15.663ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
134162024-11-14T04:59:15.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
134172024-11-14T04:59:15.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134182024-11-14T04:59:15.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
134192024-11-14T04:59:15.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
134202024-11-14T04:59:15.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
134212024-11-14T04:59:15.671ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 471521d2-c589-429d-b7c5-a658396cfcf6 dns_zone_name = oxide.test found_so_far = 1 version = 2
134222024-11-14T04:59:15.671ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
134232024-11-14T04:59:15.671ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 04:46:18.039899 UTC dns_group = external generation = 2
134242024-11-14T04:59:15.671ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 357.187633ms iteration = 15 status = {"generation":2}
134252024-11-14T04:59:15.676ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
134262024-11-14T04:59:15.676ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
134272024-11-14T04:59:15.676ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
134282024-11-14T04:59:15.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
134292024-11-14T04:59:15.680ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134302024-11-14T04:59:15.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
134312024-11-14T04:59:15.684ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
134322024-11-14T04:59:15.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
134332024-11-14T04:59:15.687ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 823c13bf-5aec-4696-b1d5-c0aaf2fd3470 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
134342024-11-14T04:59:15.687ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
134352024-11-14T04:59:15.687ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 04:46:16.023652 UTC dns_group = internal generation = 1
134362024-11-14T04:59:15.687ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 372.621271ms iteration = 15 status = {"generation":1}
134372024-11-14T04:59:15.694ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
134382024-11-14T04:59:15.694ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
134392024-11-14T04:59:15.695ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab9450a0-b4c9-49f5-b1db-9afe13dd89a3", "content-length": "26", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
134402024-11-14T04:59:15.695ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
134412024-11-14T04:59:15.697ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
134422024-11-14T04:59:15.698ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 383.6353ms iteration = 30 status = {"physical_disks_added":0}
134432024-11-14T04:59:15.705ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
134442024-11-14T04:59:15.705ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b22974b9-43d6-49f2-ba31-c0c20060dba6", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
134452024-11-14T04:59:15.705ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
134462024-11-14T04:59:15.708ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "14681e35-54be-4002-8b09-5fcf70f26f1a", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
134472024-11-14T04:59:15.708ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
134482024-11-14T04:59:15.708ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
134492024-11-14T04:59:15.708ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 393.817844ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
134502024-11-14T04:59:15.709ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
134512024-11-14T04:59:15.709ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 395.02276ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
134522024-11-14T04:59:15.714ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T04:46:10.019708Z, time_modified: 2024-11-14T04:46:10.019708Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 992dae43-7740-49b5-814b-c5341916fc63, db_silo: Silo { identity: SiloIdentity { id: 992dae43-7740-49b5-814b-c5341916fc63, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T04:46:17.991199Z, time_modified: 2024-11-14T04:46:17.991199Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f" }] }) }
134532024-11-14T04:59:15.714ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 399.447578ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]}},"default_endpoint":{"silo_id":"992dae43-7740-49b5-814b-c5341916fc63","tls_certs":["b8a326130f7feb180d822bab9229e903b08dd160da401fae1aa501ed8531f09f"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
134542024-11-14T04:59:15.745ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
134552024-11-14T04:59:15.837ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
134562024-11-14T04:59:15.837ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e05eed5-763c-410f-a09d-26e965ee70b1", "content-length": "691", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
134572024-11-14T04:59:15.837ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
134582024-11-14T04:59:15.837ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
134592024-11-14T04:59:15.837ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
134602024-11-14T04:59:15.839ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 524.913461ms iteration = 27 status = {}
134612024-11-14T04:59:15.869ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
134622024-11-14T04:59:15.870ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "72d107c3-66b2-46ec-8c87-2bb4da69c7a4", "content-length": "2", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
134632024-11-14T04:59:15.871ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
134642024-11-14T04:59:15.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
134652024-11-14T04:59:15.873ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 559.193104ms iteration = 27 status = null
134662024-11-14T04:59:15.898ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
134672024-11-14T04:59:15.899ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "dbe859d8-707a-46de-8ad5-fa36eb9c5876", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
134682024-11-14T04:59:15.899ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
134692024-11-14T04:59:15.901ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "6aa5b053-968e-4416-acaa-502530d80cf0", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
134702024-11-14T04:59:15.901ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
134712024-11-14T04:59:15.903ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "01dbfab4-c254-4fc1-a041-ae49e3c3ca46", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
134722024-11-14T04:59:15.903ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
134732024-11-14T04:59:15.905ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
134742024-11-14T04:59:15.905ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 590.940452ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
134752024-11-14T04:59:15.926ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134762024-11-14T04:59:15.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134772024-11-14T04:59:15.928ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
134782024-11-14T04:59:15.928ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134792024-11-14T04:59:15.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134802024-11-14T04:59:15.937ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
134812024-11-14T04:59:15.937ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4681d20-4b4f-4030-825e-3c02baf34163", "content-length": "134", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
134822024-11-14T04:59:15.937ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
134832024-11-14T04:59:15.937ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
134842024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
134852024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
134862024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
134872024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
134882024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
134892024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
134902024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
134912024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
134922024-11-14T04:59:15.938ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
134932024-11-14T04:59:15.938ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134942024-11-14T04:59:15.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134952024-11-14T04:59:15.941ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
134962024-11-14T04:59:15.941ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ffc6f89-72c6-4dfe-affc-070f93c584e6", "content-length": "2", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
134972024-11-14T04:59:15.941ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
134982024-11-14T04:59:15.941ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9b80ec0-4a04-4d06-a115-3f7b175dee22", "content-length": "51", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
134992024-11-14T04:59:15.941ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
135002024-11-14T04:59:15.941ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
135012024-11-14T04:59:16.225ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5893f71e-bbc1-49fa-87cd-4c3e186a7b4b", "date": "Thu, 14 Nov 2024 04:59:16 GMT"} })
135022024-11-14T04:59:16.225ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
135032024-11-14T04:59:16.225ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
135042024-11-14T04:59:16.225ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "145b7191-d1b0-4dcc-947d-72da58e382d1", "content-length": "682", "date": "Thu, 14 Nov 2024 04:59:15 GMT"} })
135052024-11-14T04:59:16.225ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135062024-11-14T04:59:16.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135072024-11-14T04:59:16.229ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135082024-11-14T04:59:16.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135092024-11-14T04:59:16.232ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135102024-11-14T04:59:16.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135112024-11-14T04:59:16.234ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135122024-11-14T04:59:16.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135132024-11-14T04:59:16.241ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 926.8136ms iteration = 27 status = {}
135142024-11-14T04:59:19.925ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:48734
135152024-11-14T04:59:19.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:48734 req_id = 06a94092-bd96-40bc-afa4-5f9b3c218ea6 roles = RoleSet { roles: {} } uri = /metrics/producers
135162024-11-14T04:59:19.925ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:48734 req_id = 06a94092-bd96-40bc-afa4-5f9b3c218ea6 resource = Database result = Ok(()) uri = /metrics/producers
135172024-11-14T04:59:19.961ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = e5586e29-e301-4b24-9a2c-39545da75716
135182024-11-14T04:59:19.961ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36285 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:48734 req_id = 06a94092-bd96-40bc-afa4-5f9b3c218ea6 response_code = 201 uri = /metrics/producers
135192024-11-14T04:59:20.256ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1074 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 099116e8-22a8-4943-8488-7a796ca35371 response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
135202024-11-14T04:59:25.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
135212024-11-14T04:59:25.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
135222024-11-14T04:59:25.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135232024-11-14T04:59:25.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135242024-11-14T04:59:25.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135252024-11-14T04:59:25.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
135262024-11-14T04:59:25.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135272024-11-14T04:59:25.344ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135282024-11-14T04:59:25.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
135292024-11-14T04:59:25.346ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.008737ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
135302024-11-14T04:59:25.346ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
135312024-11-14T04:59:25.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135322024-11-14T04:59:25.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135332024-11-14T04:59:25.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135342024-11-14T04:59:25.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135352024-11-14T04:59:25.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135362024-11-14T04:59:25.351ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
135372024-11-14T04:59:25.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135382024-11-14T04:59:25.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135392024-11-14T04:59:25.363ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
135402024-11-14T04:59:25.363ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.146943ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
135412024-11-14T04:59:30.956ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:38102
135422024-11-14T04:59:30.956ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38102 req_id = a0c526f8-ae3a-43a6-98b9-6f1662a0ddb9 roles = RoleSet { roles: {} } uri = /metrics/producers
135432024-11-14T04:59:30.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38102 req_id = a0c526f8-ae3a-43a6-98b9-6f1662a0ddb9 resource = Database result = Ok(()) uri = /metrics/producers
135442024-11-14T04:59:30.994ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = f4c8f330-91d6-4976-b637-c72f900d8c87
135452024-11-14T04:59:30.994ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37746 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38102 req_id = a0c526f8-ae3a-43a6-98b9-6f1662a0ddb9 response_code = 201 uri = /metrics/producers
135462024-11-14T04:59:32.786ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:52952
135472024-11-14T04:59:32.786ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52952 req_id = 8b5348f0-1f71-495f-b1f1-a745df9e0a61 roles = RoleSet { roles: {} } uri = /metrics/producers
135482024-11-14T04:59:32.787ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52952 req_id = 8b5348f0-1f71-495f-b1f1-a745df9e0a61 resource = Database result = Ok(()) uri = /metrics/producers
135492024-11-14T04:59:32.825ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = 18d4ee8d-1d67-44aa-96f9-89dfcadba34d
135502024-11-14T04:59:32.825ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38798 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52952 req_id = 8b5348f0-1f71-495f-b1f1-a745df9e0a61 response_code = 201 uri = /metrics/producers
135512024-11-14T04:59:35.256ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1085 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 4669fc87-87ae-47a4-a1b6-433d7ba823eb response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
135522024-11-14T04:59:35.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
135532024-11-14T04:59:35.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135542024-11-14T04:59:35.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
135552024-11-14T04:59:35.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135562024-11-14T04:59:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135572024-11-14T04:59:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
135582024-11-14T04:59:35.341ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135592024-11-14T04:59:35.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
135602024-11-14T04:59:35.342ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.415573ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
135612024-11-14T04:59:35.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135622024-11-14T04:59:35.345ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
135632024-11-14T04:59:35.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135642024-11-14T04:59:35.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135652024-11-14T04:59:35.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135662024-11-14T04:59:35.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135672024-11-14T04:59:35.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135682024-11-14T04:59:35.351ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
135692024-11-14T04:59:35.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135702024-11-14T04:59:35.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
135712024-11-14T04:59:35.362ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
135722024-11-14T04:59:35.362ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 47.620809ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
135732024-11-14T04:59:40.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = e65802f3-04d7-4048-b940-129fa8b319d6 roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
135742024-11-14T04:59:40.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = e65802f3-04d7-4048-b940-129fa8b319d6 resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
135752024-11-14T04:59:40.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3297 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = e65802f3-04d7-4048-b940-129fa8b319d6 response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?sort_by=id_ascending
135762024-11-14T04:59:40.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 24d3dfe5-1328-4fab-b235-1c38bf5267ce roles = RoleSet { roles: {} } uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
135772024-11-14T04:59:40.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 24d3dfe5-1328-4fab-b235-1c38bf5267ce resource = Database result = Ok(()) uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
135782024-11-14T04:59:40.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2741 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53964 req_id = 24d3dfe5-1328-4fab-b235-1c38bf5267ce response_code = 200 uri = /metrics/collectors/459b411b-9021-43ae-8875-254e55794157/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM4ZjMzMC05MWQ2LTQ5NzYtYjYzNy1jNzJmOTAwZDhjODcifX0%3D
135792024-11-14T04:59:44.607ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:46236
135802024-11-14T04:59:44.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:46236 req_id = d4b73d2d-46ed-4cad-997a-4f2a5a86a1dd roles = RoleSet { roles: {} } uri = /metrics/producers
135812024-11-14T04:59:44.608ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:46236 req_id = d4b73d2d-46ed-4cad-997a-4f2a5a86a1dd resource = Database result = Ok(()) uri = /metrics/producers
135822024-11-14T04:59:44.617ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = bc37b1eb-3677-438e-a276-b6d3e6fe3f8e
135832024-11-14T04:59:44.617ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9612 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:46236 req_id = d4b73d2d-46ed-4cad-997a-4f2a5a86a1dd response_code = 201 uri = /metrics/producers
135842024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
135852024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
135862024-11-14T04:59:45.314ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
135872024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135882024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
135892024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
135902024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
135912024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
135922024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
135932024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
135942024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
135952024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
135962024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
135972024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
135982024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
135992024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
136002024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
136012024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
136022024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
136032024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
136042024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
136052024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
136062024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
136072024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
136082024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
136092024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
136102024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
136112024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
136122024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
136132024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
136142024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
136152024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
136162024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
136172024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
136182024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
136192024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
136202024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
136212024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
136222024-11-14T04:59:45.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
136232024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
136242024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
136252024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
136262024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
136272024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
136282024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
136292024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
136302024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
136312024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
136322024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
136332024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
136342024-11-14T04:59:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
136352024-11-14T04:59:45.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136362024-11-14T04:59:45.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136372024-11-14T04:59:45.317ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136382024-11-14T04:59:45.317ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.44484ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
136392024-11-14T04:59:45.317ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136402024-11-14T04:59:45.318ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1764, tv_nsec: 97072309 } })
136412024-11-14T04:59:45.318ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
136422024-11-14T04:59:45.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
136432024-11-14T04:59:45.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
136442024-11-14T04:59:45.319ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
136452024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
136462024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
136472024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
136482024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
136492024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
136502024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
136512024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136522024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
136532024-11-14T04:59:45.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
136542024-11-14T04:59:45.325ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
136552024-11-14T04:59:45.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136562024-11-14T04:59:45.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
136572024-11-14T04:59:45.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136582024-11-14T04:59:45.328ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1764, tv_nsec: 106372368 } })
136592024-11-14T04:59:45.328ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
136602024-11-14T04:59:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
136612024-11-14T04:59:45.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
136622024-11-14T04:59:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
136632024-11-14T04:59:45.329ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
136642024-11-14T04:59:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
136652024-11-14T04:59:45.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
136662024-11-14T04:59:45.337ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 22.367728ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
136672024-11-14T04:59:45.339ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
136682024-11-14T04:59:45.339ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
136692024-11-14T04:59:45.339ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
136702024-11-14T04:59:45.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
136712024-11-14T04:59:45.341ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
136722024-11-14T04:59:45.341ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136732024-11-14T04:59:45.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
136742024-11-14T04:59:45.343ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
136752024-11-14T04:59:45.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
136762024-11-14T04:59:45.344ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
136772024-11-14T04:59:45.344ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
136782024-11-14T04:59:45.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
136792024-11-14T04:59:45.346ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
136802024-11-14T04:59:45.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
136812024-11-14T04:59:45.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136822024-11-14T04:59:45.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
136832024-11-14T04:59:45.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136842024-11-14T04:59:45.354ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c073d51c-abaa-48df-abb6-2f053e9c1bec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1764, tv_nsec: 132861736 } })
136852024-11-14T04:59:45.354ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
136862024-11-14T04:59:45.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
136872024-11-14T04:59:45.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136882024-11-14T04:59:45.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
136892024-11-14T04:59:45.371ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
136902024-11-14T04:59:45.371ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.255199ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
136912024-11-14T04:59:45.375ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 60.162302ms iteration = 28 status = {"errors":[],"records_set_to_done":[]}
136922024-11-14T04:59:45.379ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
136932024-11-14T04:59:45.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
136942024-11-14T04:59:45.380ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.347981ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
136952024-11-14T04:59:45.381ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
136962024-11-14T04:59:45.381ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
136972024-11-14T04:59:45.381ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
136982024-11-14T04:59:45.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
136992024-11-14T04:59:45.381ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
137002024-11-14T04:59:45.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
137012024-11-14T04:59:45.384ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137022024-11-14T04:59:45.384ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137032024-11-14T04:59:45.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137042024-11-14T04:59:45.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
137052024-11-14T04:59:45.386ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137062024-11-14T04:59:45.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137072024-11-14T04:59:45.387ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137082024-11-14T04:59:45.387ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
137092024-11-14T04:59:45.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137102024-11-14T04:59:45.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
137112024-11-14T04:59:45.391ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 76.6502ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
137122024-11-14T04:59:45.395ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
137132024-11-14T04:59:45.395ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 81.430068ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
137142024-11-14T04:59:45.426ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
137152024-11-14T04:59:45.426ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
137162024-11-14T04:59:45.426ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 111.798558ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
137172024-11-14T04:59:45.426ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 110.716976ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
137182024-11-14T04:59:45.429ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
137192024-11-14T04:59:45.429ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
137202024-11-14T04:59:45.429ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9aeac13f-1db3-462d-945f-507e3fac75f9", "content-length": "26", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
137212024-11-14T04:59:45.429ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
137222024-11-14T04:59:45.431ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
137232024-11-14T04:59:45.434ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137242024-11-14T04:59:45.437ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c073d51c-abaa-48df-abb6-2f053e9c1bec, lookup_type: ById(c073d51c-abaa-48df-abb6-2f053e9c1bec) } result = Ok(())
137252024-11-14T04:59:45.437ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137262024-11-14T04:59:45.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137272024-11-14T04:59:45.469ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
137282024-11-14T04:59:45.469ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
137292024-11-14T04:59:45.470ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6407819b-b764-4def-bee6-d382ce862a45", "content-length": "26", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
137302024-11-14T04:59:45.470ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
137312024-11-14T04:59:45.471ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
137322024-11-14T04:59:45.472ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 zone_address = fd00:1122:3344:101::2
137332024-11-14T04:59:45.472ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
137342024-11-14T04:59:45.473ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3cf7838-42a1-4552-ab36-3ec8fb85e10a", "content-length": "26", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
137352024-11-14T04:59:45.473ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
137362024-11-14T04:59:45.474ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f971eb12-0fea-4084-96d3-74198bddd289
137372024-11-14T04:59:45.484ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
137382024-11-14T04:59:45.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
137392024-11-14T04:59:45.501ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
137402024-11-14T04:59:45.501ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
137412024-11-14T04:59:45.501ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cba77e39-05e8-40f9-9f26-a0dfa0d42fad", "content-length": "372", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
137422024-11-14T04:59:45.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137432024-11-14T04:59:45.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137442024-11-14T04:59:45.506ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 192.538026ms iteration = 28 status = 0
137452024-11-14T04:59:45.514ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 199.976922ms iteration = 31 status = {"physical_disks_added":0}
137462024-11-14T04:59:45.522ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 207.850109ms iteration = 28 status = {}
137472024-11-14T04:59:45.534ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
137482024-11-14T04:59:45.534ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
137492024-11-14T04:59:45.534ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "942857b4-2d23-4088-8b3b-fa73f8310ca4", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
137502024-11-14T04:59:45.534ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137512024-11-14T04:59:45.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137522024-11-14T04:59:45.538ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137532024-11-14T04:59:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137542024-11-14T04:59:45.540ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137552024-11-14T04:59:45.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137562024-11-14T04:59:45.544ZINFOnexus (BackgroundTasks): VPC route manager sled c073d51c-abaa-48df-abb6-2f053e9c1bec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
137572024-11-14T04:59:45.544ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
137582024-11-14T04:59:45.544ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "eabccb3f-1f2b-47fb-8650-e2563dc65169", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
137592024-11-14T04:59:45.546ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 232.697268ms iteration = 29 status = {}
137602024-11-14T04:59:45.547ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 231.785642ms iteration = 28 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
137612024-11-14T04:59:45.559ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
137622024-11-14T04:59:45.559ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "f691656c-0ecd-4806-bb32-6cdd87befcc2", "content-length": "2", "date": "Thu, 14 Nov 2024 04:59:44 GMT"} })
137632024-11-14T04:59:45.561ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
137642024-11-14T04:59:45.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
137652024-11-14T04:59:45.563ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 248.878377ms iteration = 28 status = null
137662024-11-14T04:59:45.601ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
137672024-11-14T04:59:45.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
137682024-11-14T04:59:45.603ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137692024-11-14T04:59:45.605ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
137702024-11-14T04:59:45.605ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
137712024-11-14T04:59:45.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
137722024-11-14T04:59:45.617ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137732024-11-14T04:59:45.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137742024-11-14T04:59:45.620ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 7a4d1fda-a07d-4bbb-b64b-95a0877b4ce5, rack_id: f971eb12-0fea-4084-96d3-74198bddd289, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cd4e4950-ced0-45b6-a0d9-7018602e099c) } rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
137752024-11-14T04:59:45.620ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137762024-11-14T04:59:45.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137772024-11-14T04:59:45.633ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
137782024-11-14T04:59:45.633ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea697ffd-7635-4afb-af54-3517b4c16927", "content-length": "134", "date": "Thu, 14 Nov 2024 04:59:44 GMT"} })
137792024-11-14T04:59:45.633ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
137802024-11-14T04:59:45.633ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cd4e4950-ced0-45b6-a0d9-7018602e099c
137812024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
137822024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
137832024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
137842024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
137852024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
137862024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
137872024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
137882024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
137892024-11-14T04:59:45.634ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f971eb12-0fea-4084-96d3-74198bddd289
137902024-11-14T04:59:45.634ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137912024-11-14T04:59:45.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137922024-11-14T04:59:45.636ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
137932024-11-14T04:59:45.637ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "49001ab8-bf02-4332-8b9f-45fa010b9abb", "content-length": "2", "date": "Thu, 14 Nov 2024 04:59:44 GMT"} })
137942024-11-14T04:59:45.637ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
137952024-11-14T04:59:45.637ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b40e111e-3b2a-48cb-b566-412da1a2d222", "content-length": "51", "date": "Thu, 14 Nov 2024 04:59:44 GMT"} })
137962024-11-14T04:59:45.637ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch_location = Switch0
137972024-11-14T04:59:45.637ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
137982024-11-14T04:59:45.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137992024-11-14T04:59:45.645ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
138002024-11-14T04:59:45.645ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
138012024-11-14T04:59:45.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
138022024-11-14T04:59:45.734ZDEBGnexus (BackgroundTasks): client request SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
138032024-11-14T04:59:45.748ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "67c32e24-f09a-4aa7-9c17-fc224e2237b2", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
138042024-11-14T04:59:45.748ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f971eb12-0fea-4084-96d3-74198bddd289 switch = Switch0
138052024-11-14T04:59:45.748ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f971eb12-0fea-4084-96d3-74198bddd289 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
138062024-11-14T04:59:45.753ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f971eb12-0fea-4084-96d3-74198bddd289 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "33891cde-29a5-4adc-b730-51134b384a0c", "content-length": "682", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
138072024-11-14T04:59:45.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138082024-11-14T04:59:45.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138092024-11-14T04:59:45.753ZDEBGnexus (BackgroundTasks): client response SledAgent = c073d51c-abaa-48df-abb6-2f053e9c1bec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e1ea0dd-d1a9-4092-8e4d-319811461a40", "content-length": "691", "date": "Thu, 14 Nov 2024 04:59:45 GMT"} })
138102024-11-14T04:59:45.753ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
138112024-11-14T04:59:45.753ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
138122024-11-14T04:59:45.753ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
138132024-11-14T04:59:45.756ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 441.639069ms iteration = 28 status = {}
138142024-11-14T04:59:45.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138152024-11-14T04:59:45.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138162024-11-14T04:59:45.760ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138172024-11-14T04:59:45.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138182024-11-14T04:59:45.762ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138192024-11-14T04:59:45.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138202024-11-14T04:59:45.769ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 454.678086ms iteration = 28 status = {}
138212024-11-14T04:59:49.352ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:55062
138222024-11-14T04:59:49.352ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:55062 req_id = 6125e970-b1fd-4135-af98-1854066d391c roles = RoleSet { roles: {} } uri = /metrics/producers
138232024-11-14T04:59:49.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:55062 req_id = 6125e970-b1fd-4135-af98-1854066d391c resource = Database result = Ok(()) uri = /metrics/producers
138242024-11-14T04:59:49.405ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = c073d51c-abaa-48df-abb6-2f053e9c1bec
138252024-11-14T04:59:49.405ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53016 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:55062 req_id = 6125e970-b1fd-4135-af98-1854066d391c response_code = 201 uri = /metrics/producers
138262024-11-14T04:59:50.257ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1573 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 94a35c1b-28a2-447b-9ce2-6f2ddad6ceba response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
138272024-11-14T04:59:55.314ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
138282024-11-14T04:59:55.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138292024-11-14T04:59:55.314ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
138302024-11-14T04:59:55.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
138312024-11-14T04:59:55.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138322024-11-14T04:59:55.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
138332024-11-14T04:59:55.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138342024-11-14T04:59:55.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
138352024-11-14T04:59:55.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138362024-11-14T04:59:55.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138372024-11-14T04:59:55.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138382024-11-14T04:59:55.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138392024-11-14T04:59:55.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138402024-11-14T04:59:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
138412024-11-14T04:59:55.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138422024-11-14T04:59:55.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138432024-11-14T04:59:55.335ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
138442024-11-14T04:59:55.335ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.508723ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
138452024-11-14T04:59:55.342ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
138462024-11-14T04:59:55.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
138472024-11-14T04:59:55.370ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 55.810127ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
138482024-11-14T05:00:05.256ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1200 local_addr = [fd00:1122:3344:101::b]:49209 method = GET remote_addr = [fd00:1122:3344:101::d]:55473 req_id = 4623960a-272c-4e50-b3c2-edbd7fb0f21f response_code = 200 uri = /13db289d-f9b6-41e9-b3bd-5ab4f3105694
138492024-11-14T05:00:05.313ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
138502024-11-14T05:00:05.313ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
138512024-11-14T05:00:05.313ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
138522024-11-14T05:00:05.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138532024-11-14T05:00:05.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138542024-11-14T05:00:05.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
138552024-11-14T05:00:05.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138562024-11-14T05:00:05.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
138572024-11-14T05:00:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
138582024-11-14T05:00:05.317ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.511712ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
138592024-11-14T05:00:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
138602024-11-14T05:00:05.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138612024-11-14T05:00:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138622024-11-14T05:00:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138632024-11-14T05:00:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138642024-11-14T05:00:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138652024-11-14T05:00:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 601475e8-5db8-414f-91f2-d6124fa0dcd7, lookup_type: ById(601475e8-5db8-414f-91f2-d6124fa0dcd7) } result = Ok(())
138662024-11-14T05:00:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138672024-11-14T05:00:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
138682024-11-14T05:00:05.344ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 original_time_created = 2024-11-14 04:46:04.796077 UTC target_id = 601475e8-5db8-414f-91f2-d6124fa0dcd7 time_created = 2024-11-14 04:46:04.796077 UTC
138692024-11-14T05:00:05.344ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.465432ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"601475e8-5db8-414f-91f2-d6124fa0dcd7","time_created":"2024-11-14T04:46:04.796077Z"}
138702024-11-14T05:00:11.835ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:52344
138712024-11-14T05:00:11.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52344 req_id = d139ca6b-76e3-43ee-a1b2-5750c3467ee2 roles = RoleSet { roles: {} } uri = /metrics/producers
138722024-11-14T05:00:11.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52344 req_id = d139ca6b-76e3-43ee-a1b2-5750c3467ee2 resource = Database result = Ok(()) uri = /metrics/producers
138732024-11-14T05:00:11.846ZINFOnexus: assigned collector to new producer collector_id = 459b411b-9021-43ae-8875-254e55794157 file = nexus/src/app/oximeter.rs:76 producer_id = b584ea61-acb3-47a3-a173-0d03f6cae9c9
138742024-11-14T05:00:11.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10565 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52344 req_id = d139ca6b-76e3-43ee-a1b2-5750c3467ee2 response_code = 201 uri = /metrics/producers